feat(bdt): Python parser core + CLI subcommands @W-22196528@

Browse Source

Adds scripts/bdt_analyze.py — a generic, stdlib-only DAG parser and
query CLI for Salesforce Data Cloud BDT JSON. 1,379 lines.

Parser primitives:
- DataTransform.from_path / from_dict — accepts three input shapes:
  editor export ({version, nodes, ui, ...}), Connect API
  single-definition ({name, label, type, definition: {...}}), and
  Connect API multi-definition ({name, definitions: [{name, label,
  type, definition}, ...]}).
- Node dataclass with ui_label / ui_description fallback resolution.
- roots() / sinks() / topo_order() — Kahn's algorithm with
  deterministic tie-break for reproducible output.
- upstream() / downstream() traversal resilient to broken refs and
  cycles (does not infinite-loop on self-edges or cycles).
- broken_references(), fields_produced(), fields_consumed(),
  _scrape_field_refs() heuristics for field-trace discovery.

CLI (argparse, 10 subcommands, each supports --json for machine
output):
- summary      — node counts, source/sink counts, stage totals.
- sources      — list source nodes (no upstream).
- outputs      — list sink nodes (no downstream).
- stages       — topologically ordered stages.
- nodes        — flat node listing with labels.
- node <name>  — per-node detail (action, inputs, outputs, fields,
                 UI label/description).
- lineage <node>       — upstream + downstream chain from a node.
- field-trace <field>  — which nodes produce/consume a given field.
- formula <node>       — extract formulas/expressions from a node.
- definitions          — lists definitions in multi-definition
                         payloads; every other subcommand accepts
                         --definition N (default 0) to route into a
                         specific definition within the payload.

Error contract:
- Exit 0 on success, 2 on unknown node/field, 3 on malformed input.
- BdtInputError (exit 3) and BdtNotFoundError (exit 2) classes
  centralize error handling so the CLI shell stays thin.
- Field-trace narrowing refinements prevent false positives from
  substring matches in formula bodies.
- Upstream/downstream walkers harden against broken refs discovered
  during internal-BDT audit.

Design invariants:
- Python owns truth (parsing, DAG math, field discovery). LLM owns
  narrative (explaining what the structure means to a user).
- No external dependencies — stdlib only: argparse, json, pathlib,
  re, sys, collections, dataclasses, typing.
- Output size budgets: every subcommand caps its default-mode output
  so summaries fit in a single LLM context window; --json dumps
  everything for agents that need raw data.

@W-22196528@

...

This commit is contained in:

Gaurav Bajpai 2026-04-23 22:44:59 +05:30

parent 9be9667779

commit 9f1603ed2f

No known key found for this signature in database

GPG Key ID: 4EE015E4DD141C47

1 changed files with 1379 additions and 0 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1379

skills/explaining-batch-data-transform/scripts/bdt_analyze.py Normal file

View File

File diff suppressed because it is too large Load Diff