Adds the curated reference library the skill loads on demand, plus
four synthetic sample BDTs used by docs, tests, and LLM-mode demos.
references/ (4 curated Markdown files):
- bdt-reference.md — top-level BDT JSON anatomy: envelope,
nodes, edges, UI layer, definitions,
businessType semantics. Cites the core-262
upstream JSON schema and Connect API spec.
- bdt-node-catalog.md — every node type (DMO Source, DMO Sink,
Filter, Join, Union, Aggregate, Window,
Formula, Split, Append, etc.) with its
required/optional fields and typical
usage. Audited against core-262 enums.
- bdt-function-catalog.md — the expression-language function surface
(string, numeric, date, conditional,
aggregate). Grouped by category with
signature + one-line semantics.
- bdt-window-functions.md — windowing operators (ROW_NUMBER, RANK,
LEAD/LAG, running aggregates) with PARTITION
BY / ORDER BY grammar and gotchas.
assets/sample_bdts/ (4 synthetic, dependency-free BDTs):
- minimal_dmo_to_dmo.json — smallest valid BDT (1 source, 1 sink).
- joins_and_filters.json — join + filter composition.
- window_and_aggregate.json — window function + aggregate in one graph.
- append_and_split.json — append-then-split branching topology.
Grounding rules enforced in this commit:
- Every claim in references/ cites an upstream source (core-262 JSON
schema, Connect API reference, or the Data Cloud BDT editor spec).
No speculative content.
- No raw DITA or internal-only documentation is shipped; references
are synthesized from public-facing material.
- BusinessTypeEnum values use the canonical camelCase casing from
core-262 (case-cleanup fix included here).
- Sample BDTs are original synthetic fixtures, not redacted customer
data. Each is small enough to read end-to-end.
@W-22196528@
Adds scripts/bdt_analyze.py — a generic, stdlib-only DAG parser and
query CLI for Salesforce Data Cloud BDT JSON. 1,379 lines.
Parser primitives:
- DataTransform.from_path / from_dict — accepts three input shapes:
editor export ({version, nodes, ui, ...}), Connect API
single-definition ({name, label, type, definition: {...}}), and
Connect API multi-definition ({name, definitions: [{name, label,
type, definition}, ...]}).
- Node dataclass with ui_label / ui_description fallback resolution.
- roots() / sinks() / topo_order() — Kahn's algorithm with
deterministic tie-break for reproducible output.
- upstream() / downstream() traversal resilient to broken refs and
cycles (does not infinite-loop on self-edges or cycles).
- broken_references(), fields_produced(), fields_consumed(),
_scrape_field_refs() heuristics for field-trace discovery.
CLI (argparse, 10 subcommands, each supports --json for machine
output):
- summary — node counts, source/sink counts, stage totals.
- sources — list source nodes (no upstream).
- outputs — list sink nodes (no downstream).
- stages — topologically ordered stages.
- nodes — flat node listing with labels.
- node <name> — per-node detail (action, inputs, outputs, fields,
UI label/description).
- lineage <node> — upstream + downstream chain from a node.
- field-trace <field> — which nodes produce/consume a given field.
- formula <node> — extract formulas/expressions from a node.
- definitions — lists definitions in multi-definition
payloads; every other subcommand accepts
--definition N (default 0) to route into a
specific definition within the payload.
Error contract:
- Exit 0 on success, 2 on unknown node/field, 3 on malformed input.
- BdtInputError (exit 3) and BdtNotFoundError (exit 2) classes
centralize error handling so the CLI shell stays thin.
- Field-trace narrowing refinements prevent false positives from
substring matches in formula bodies.
- Upstream/downstream walkers harden against broken refs discovered
during internal-BDT audit.
Design invariants:
- Python owns truth (parsing, DAG math, field discovery). LLM owns
narrative (explaining what the structure means to a user).
- No external dependencies — stdlib only: argparse, json, pathlib,
re, sys, collections, dataclasses, typing.
- Output size budgets: every subcommand caps its default-mode output
so summaries fit in a single LLM context window; --json dumps
everything for agents that need raw data.
@W-22196528@
Scaffolds the explaining-batch-data-transform skill directory and delivers
the full, production-grade SKILL.md body. 194 lines.
Directory layout established (skills/explaining-batch-data-transform/):
- SKILL.md — the entry point this commit adds
- scripts/ — parser lands in commit 2
- references/, assets/ — reference docs + samples land in commit 3
- tests/ — unit tests + fixtures land in commit 4
SKILL.md frontmatter:
- name, description, tier, and owner fields matching the library's CI
validator schema.
- Description covers the full production surface: "Explain, audit, and
trace Salesforce Data Cloud Batch Data Transform (BDT) JSON — sources,
sinks, stages, lineage, field traces, formulas, multi-definition
payloads, and dual input shapes (editor export + Connect API)."
SKILL.md body (12 sections):
1. When to use this skill
2. Inputs you accept (editor export, Connect API single-definition,
Connect API multi-definition payloads)
3. Mode A — structural survey (summary / sources / outputs / stages)
4. Mode B — per-node explanation (nodes / node)
5. Mode C — lineage + field trace
6. Mode D — formulas and windows
7. Question routing table (user phrasing -> CLI subcommand)
8. Multi-definition payload handling (--definition flag semantics)
9. Troubleshooting (malformed JSON, unknown node refs, broken upstream)
10. Non-goals as explicit "Do not" directives (no execution, no schema
inference beyond what the JSON states, no speculation about runtime)
11. References + sample BDTs index
12. Exit-code + error-class contract surfaced from bdt_analyze.py
Structure passes the repo's skill-validator (npm run validate:skills):
single top-level H1, required sections present, frontmatter schema
aligned, no broken internal links.
@W-22196528@
