Adds scripts/bdt_analyze.py — a generic, stdlib-only DAG parser and
query CLI for Salesforce Data Cloud BDT JSON. 1,379 lines.
Parser primitives:
- DataTransform.from_path / from_dict — accepts three input shapes:
editor export ({version, nodes, ui, ...}), Connect API
single-definition ({name, label, type, definition: {...}}), and
Connect API multi-definition ({name, definitions: [{name, label,
type, definition}, ...]}).
- Node dataclass with ui_label / ui_description fallback resolution.
- roots() / sinks() / topo_order() — Kahn's algorithm with
deterministic tie-break for reproducible output.
- upstream() / downstream() traversal resilient to broken refs and
cycles (does not infinite-loop on self-edges or cycles).
- broken_references(), fields_produced(), fields_consumed(),
_scrape_field_refs() heuristics for field-trace discovery.
CLI (argparse, 10 subcommands, each supports --json for machine
output):
- summary — node counts, source/sink counts, stage totals.
- sources — list source nodes (no upstream).
- outputs — list sink nodes (no downstream).
- stages — topologically ordered stages.
- nodes — flat node listing with labels.
- node <name> — per-node detail (action, inputs, outputs, fields,
UI label/description).
- lineage <node> — upstream + downstream chain from a node.
- field-trace <field> — which nodes produce/consume a given field.
- formula <node> — extract formulas/expressions from a node.
- definitions — lists definitions in multi-definition
payloads; every other subcommand accepts
--definition N (default 0) to route into a
specific definition within the payload.
Error contract:
- Exit 0 on success, 2 on unknown node/field, 3 on malformed input.
- BdtInputError (exit 3) and BdtNotFoundError (exit 2) classes
centralize error handling so the CLI shell stays thin.
- Field-trace narrowing refinements prevent false positives from
substring matches in formula bodies.
- Upstream/downstream walkers harden against broken refs discovered
during internal-BDT audit.
Design invariants:
- Python owns truth (parsing, DAG math, field discovery). LLM owns
narrative (explaining what the structure means to a user).
- No external dependencies — stdlib only: argparse, json, pathlib,
re, sys, collections, dataclasses, typing.
- Output size budgets: every subcommand caps its default-mode output
so summaries fit in a single LLM context window; --json dumps
everything for agents that need raw data.
@W-22196528@
Scaffolds the explaining-batch-data-transform skill directory and delivers
the full, production-grade SKILL.md body. 194 lines.
Directory layout established (skills/explaining-batch-data-transform/):
- SKILL.md — the entry point this commit adds
- scripts/ — parser lands in commit 2
- references/, assets/ — reference docs + samples land in commit 3
- tests/ — unit tests + fixtures land in commit 4
SKILL.md frontmatter:
- name, description, tier, and owner fields matching the library's CI
validator schema.
- Description covers the full production surface: "Explain, audit, and
trace Salesforce Data Cloud Batch Data Transform (BDT) JSON — sources,
sinks, stages, lineage, field traces, formulas, multi-definition
payloads, and dual input shapes (editor export + Connect API)."
SKILL.md body (12 sections):
1. When to use this skill
2. Inputs you accept (editor export, Connect API single-definition,
Connect API multi-definition payloads)
3. Mode A — structural survey (summary / sources / outputs / stages)
4. Mode B — per-node explanation (nodes / node)
5. Mode C — lineage + field trace
6. Mode D — formulas and windows
7. Question routing table (user phrasing -> CLI subcommand)
8. Multi-definition payload handling (--definition flag semantics)
9. Troubleshooting (malformed JSON, unknown node refs, broken upstream)
10. Non-goals as explicit "Do not" directives (no execution, no schema
inference beyond what the JSON states, no speculation about runtime)
11. References + sample BDTs index
12. Exit-code + error-class contract surfaced from bdt_analyze.py
Structure passes the repo's skill-validator (npm run validate:skills):
single top-level H1, required sections present, frontmatter schema
aligned, no broken internal links.
@W-22196528@
