kris/afv-library
1
0
Fork 0
mirror of https://github.com/forcedotcom/afv-library.git synced 2026-08-09 00:42:46 +08:00
Code Issues Actions Packages Projects Releases Wiki Activity
178 Commits 9 Branches 53 Tags 31 MiB
db1aca7221
Commit Graph

5 Commits

Author SHA1 Message Date
Gaurav Bajpai
db1aca7221
fix: address PRizm review findings from internal port PR @W-22196528@ 
Addresses 4 critical findings surfaced during PRizm code review of the
internal plugin port of this skill (internal PR #19). Applying the same
fixes here keeps the external canonical source and the internal port in
sync.

1. Test cleanup discipline: switch `test_invalid_json_raises` from
   `try/finally` to `self.addCleanup(p.unlink, missing_ok=True)` — the
   unittest-idiomatic way to guarantee temp-file cleanup regardless of
   how the test exits.

2. `floor()` description in bdt-function-catalog.md: the old row was
   self-contradictory ("toward zero" AND "toward next integer up" in
   the same cell). Replace with a single coherent definition: rounds
   toward negative infinity; for negatives rounds away from zero
   (e.g., `floor(-2.3) = -3`).

3. Split-node documentation in bdt-node-catalog.md: the old doc claimed
   `split` routes rows into downstream branches via `branches[]` with
   per-branch predicates. That is not the canonical schema. Per
   `SplitParametersInputRepresentation` in core-262-public, `split` is
   a string-splitting operation: one `sourceField` + `delimiter` →
   N `targetFields` (one row in, one row out; columns added). Rewrote
   the section with the correct parameters, lineage effect, gotchas,
   and a canonical example. Row-routing belongs in `filter` nodes.

4. Sample `assets/sample_bdts/append_and_split.json`: the old sample
   used the invented `branches[]` shape AND routed the same split into
   two downstream outputs that each expected different rows — which is
   not how `split` works. Rewrote the sample so:
     - `appendV2` unions two order sources (unchanged intent).
     - `split` uses canonical `{sourceField, delimiter, targetFields}`
       splitting `CustomerFullName__c` into first + last name columns.
     - One downstream output consumes the new columns (removes the
       fake two-branch fan-out).

Tests: 92/92 passing. Sample parses and runs through `bdt_analyze.py
summary` cleanly (5 nodes: 2 load + 1 appendV2 + 1 split + 1 outputD360).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-24 10:19:36 +05:30
Gaurav Bajpai
fc4d14bce0
test(bdt): unit tests + adversarial fixtures @W-22196528@ 
Adds the full test suite for bdt_analyze.py: 92 unit tests backed by
9 JSON fixtures (4 normative + 5 adversarial). All tests stdlib-only,
runnable via `python3 -m unittest tests.test_bdt_analyze`.

tests/test_bdt_analyze.py coverage:

DAG primitives:
- roots() / sinks() / topo_order() on minimal + branching graphs.
- upstream() / downstream() reachability correctness.
- Cycle tolerance: topo_order + traversals do not infinite-loop or
  raise when the graph contains a cycle; cycle is reported via
  broken_references-adjacent signals.
- Broken-reference tolerance: nodes referencing non-existent upstreams
  are handled gracefully; broken_references() enumerates them.

CLI subcommands (happy-path + error-path for each):
- summary, sources, outputs, stages, nodes, node, lineage,
  field-trace, formula, definitions — each exercised in both
  human-readable and --json modes.
- Exit-code discipline: unknown node -> exit 2, malformed input ->
  exit 3, success -> exit 0.
- Unknown-action graceful degradation: nodes whose action isn't in
  the catalog still appear in summary/nodes output with a generic
  label rather than crashing.
- Output size budgets: default-mode outputs are asserted to stay
  within configured character caps.

Field discovery:
- fields_produced() / fields_consumed() / _scrape_field_refs()
  heuristics tested against both clean and noisy formula bodies.
- TestFieldTraceFocus — narrowing tests preventing false-positive
  substring hits in formula text.

Dual-shape input:
- TestInputShapeDetection — editor export vs. Connect API
  single-definition auto-detection.
- TestMultiDefinition — Connect API multi-definition payload:
  definitions subcommand, --definition N routing, out-of-range
  index error handling.

Fixtures (tests/fixtures/):
- minimal.json               — smallest valid editor-export BDT.
- window_and_aggregate.json  — window + aggregate composition.
- api_input_single.json      — Connect API single-definition payload.
- api_input_multi.json       — Connect API multi-definition payload.

Adversarial fixtures (tests/fixtures/adversarial/):
- cycle.json           — graph with a cycle; parser must not hang.
- broken_ref.json      — node referencing a non-existent upstream.
- unknown_action.json  — node with an action not in the catalog.
- no_ui.json           — BDT missing the UI layer entirely.
- empty_nodes.json     — valid envelope but zero nodes.

Suite status: Ran 92 tests, OK.

@W-22196528@
 2026-04-23 23:29:05 +05:30
Gaurav Bajpai
9c47f28221
feat(bdt): reference docs + sample BDTs @W-22196528@ 
Adds the curated reference library the skill loads on demand, plus
four synthetic sample BDTs used by docs, tests, and LLM-mode demos.

references/ (4 curated Markdown files):
- bdt-reference.md        — top-level BDT JSON anatomy: envelope,
                            nodes, edges, UI layer, definitions,
                            businessType semantics. Cites the core-262
                            upstream JSON schema and Connect API spec.
- bdt-node-catalog.md     — every node type (DMO Source, DMO Sink,
                            Filter, Join, Union, Aggregate, Window,
                            Formula, Split, Append, etc.) with its
                            required/optional fields and typical
                            usage. Audited against core-262 enums.
- bdt-function-catalog.md — the expression-language function surface
                            (string, numeric, date, conditional,
                            aggregate). Grouped by category with
                            signature + one-line semantics.
- bdt-window-functions.md — windowing operators (ROW_NUMBER, RANK,
                            LEAD/LAG, running aggregates) with PARTITION
                            BY / ORDER BY grammar and gotchas.

assets/sample_bdts/ (4 synthetic, dependency-free BDTs):
- minimal_dmo_to_dmo.json   — smallest valid BDT (1 source, 1 sink).
- joins_and_filters.json    — join + filter composition.
- window_and_aggregate.json — window function + aggregate in one graph.
- append_and_split.json     — append-then-split branching topology.

Grounding rules enforced in this commit:
- Every claim in references/ cites an upstream source (core-262 JSON
  schema, Connect API reference, or the Data Cloud BDT editor spec).
  No speculative content.
- No raw DITA or internal-only documentation is shipped; references
  are synthesized from public-facing material.
- BusinessTypeEnum values use the canonical camelCase casing from
  core-262 (case-cleanup fix included here).
- Sample BDTs are original synthetic fixtures, not redacted customer
  data. Each is small enough to read end-to-end.

@W-22196528@
 2026-04-23 23:29:05 +05:30
Gaurav Bajpai
9f1603ed2f
feat(bdt): Python parser core + CLI subcommands @W-22196528@ 
Adds scripts/bdt_analyze.py — a generic, stdlib-only DAG parser and
query CLI for Salesforce Data Cloud BDT JSON. 1,379 lines.

Parser primitives:
- DataTransform.from_path / from_dict — accepts three input shapes:
  editor export ({version, nodes, ui, ...}), Connect API
  single-definition ({name, label, type, definition: {...}}), and
  Connect API multi-definition ({name, definitions: [{name, label,
  type, definition}, ...]}).
- Node dataclass with ui_label / ui_description fallback resolution.
- roots() / sinks() / topo_order() — Kahn's algorithm with
  deterministic tie-break for reproducible output.
- upstream() / downstream() traversal resilient to broken refs and
  cycles (does not infinite-loop on self-edges or cycles).
- broken_references(), fields_produced(), fields_consumed(),
  _scrape_field_refs() heuristics for field-trace discovery.

CLI (argparse, 10 subcommands, each supports --json for machine
output):
- summary      — node counts, source/sink counts, stage totals.
- sources      — list source nodes (no upstream).
- outputs      — list sink nodes (no downstream).
- stages       — topologically ordered stages.
- nodes        — flat node listing with labels.
- node <name>  — per-node detail (action, inputs, outputs, fields,
                 UI label/description).
- lineage <node>       — upstream + downstream chain from a node.
- field-trace <field>  — which nodes produce/consume a given field.
- formula <node>       — extract formulas/expressions from a node.
- definitions          — lists definitions in multi-definition
                         payloads; every other subcommand accepts
                         --definition N (default 0) to route into a
                         specific definition within the payload.

Error contract:
- Exit 0 on success, 2 on unknown node/field, 3 on malformed input.
- BdtInputError (exit 3) and BdtNotFoundError (exit 2) classes
  centralize error handling so the CLI shell stays thin.
- Field-trace narrowing refinements prevent false positives from
  substring matches in formula bodies.
- Upstream/downstream walkers harden against broken refs discovered
  during internal-BDT audit.

Design invariants:
- Python owns truth (parsing, DAG math, field discovery). LLM owns
  narrative (explaining what the structure means to a user).
- No external dependencies — stdlib only: argparse, json, pathlib,
  re, sys, collections, dataclasses, typing.
- Output size budgets: every subcommand caps its default-mode output
  so summaries fit in a single LLM context window; --json dumps
  everything for agents that need raw data.

@W-22196528@
 2026-04-23 23:29:05 +05:30
Gaurav Bajpai
9be9667779
feat(bdt): scaffold skill directory + SKILL.md body @W-22196528@ 
Scaffolds the explaining-batch-data-transform skill directory and delivers
the full, production-grade SKILL.md body. 194 lines.

Directory layout established (skills/explaining-batch-data-transform/):
- SKILL.md                 — the entry point this commit adds
- scripts/                 — parser lands in commit 2
- references/, assets/     — reference docs + samples land in commit 3
- tests/                   — unit tests + fixtures land in commit 4

SKILL.md frontmatter:
- name, description, tier, and owner fields matching the library's CI
  validator schema.
- Description covers the full production surface: "Explain, audit, and
  trace Salesforce Data Cloud Batch Data Transform (BDT) JSON — sources,
  sinks, stages, lineage, field traces, formulas, multi-definition
  payloads, and dual input shapes (editor export + Connect API)."

SKILL.md body (12 sections):
1. When to use this skill
2. Inputs you accept (editor export, Connect API single-definition,
   Connect API multi-definition payloads)
3. Mode A — structural survey (summary / sources / outputs / stages)
4. Mode B — per-node explanation (nodes / node)
5. Mode C — lineage + field trace
6. Mode D — formulas and windows
7. Question routing table (user phrasing -> CLI subcommand)
8. Multi-definition payload handling (--definition flag semantics)
9. Troubleshooting (malformed JSON, unknown node refs, broken upstream)
10. Non-goals as explicit "Do not" directives (no execution, no schema
    inference beyond what the JSON states, no speculation about runtime)
11. References + sample BDTs index
12. Exit-code + error-class contract surfaced from bdt_analyze.py

Structure passes the repo's skill-validator (npm run validate:skills):
single top-level H1, required sections present, frontmatter schema
aligned, no broken internal links.

@W-22196528@
 2026-04-23 23:29:04 +05:30