afv-library/skills/investigating-agentforce-d360/scripts/tests/test_session_shape.py

96 lines
3.5 KiB
Python
Raw Normal View History

@W-22707610 feat: add investigating-agentforce-d360 skill Data Cloud 360° view of a single Agentforce session — DC-only, zero Splunk dependency. Pulls 24 STDM + GenAI DMOs via the Data Cloud Query REST API, assembles a hierarchical session tree (Interaction → Step → Generation → GatewayRequest), and renders a human-readable markdown summary with transcript + per-turn topic/action invocations + LLM generations + tool calls + audit chain. Migrated as a standalone Apache-2.0 skill from an internal hub plugin — self-contained, no sibling-skill or plugin dependencies. What this skill answers: - "Trace session <uuid>" / "Summarize what happened in <0Mw…>" - "Find escalated sessions today on Messaging in <org>" - Session discovery by time / agent / channel / outcome / conversation text when the user has no session id What it does NOT answer (use a different surface): - Design-time architecture — use investigating-agentforce-architecture - Runtime planner availability — DC alone can't tell you which topic/action was eligible for the classifier on a given turn Skill layout: - 8 Python pipeline modules (fetch_dc, assemble_dc, render_dc, discover_sessions, resolve_session, dc, storage, config) - 4 _shared helpers (paths, fs_guard, sql, __init__) with skill-scoped DATA_ROOT (~/.claude/data/investigating-agentforce-d360/) - 26 SQL templates under assets/dc/ - 27 test files (367 tests + 18 subtests, 100% passing) - 3 reference docs (artifacts.md, dc_dmo_fields.md, dc_pipeline_contract.md) - SKILL.md (sf-skills frontmatter, license: Apache-2.0, metadata.version: "1.0") - README.md (external-facing quick-start) - tools/grant_allowlist.py (idempotent first-run permission grant) - tools/archive_data_dir.sh (opt-in stop-hook tarballer) Quality gates: - pytest scripts/tests/: 367 passed + 18 subtests, 0 failures - npm run validate:skills: 62 of 62 skill(s) checked, 0 errors - Live end-to-end runs against 3 real Salesforce sessions exercising both the full-tree and STDM-lag gateway-direct render branches - 4 independent code-review rounds (correctness, security, markdown, architecture-critic) — all findings addressed Customer-data hygiene: no live tenant ids, no internal sprint markers, no hub/sibling-skill references. Synthetic fixtures look obviously synthetic (`019dface-…` UUIDs, `0MwTESTMSG…` MessagingSession ids, `00DTESTORG…` org ids, `MyAgent` placeholder agent name). Sibling skill: investigating-agentforce-architecture (PR #278) — same migration pattern, design-time metadata; complementary scope.
2026-05-28 18:57:52 +08:00
"""Tests for ``fetch_dc._classify_session_shape``.
Covers all 5 shapes with a parameterized table:
- session_not_found sessions.json returned 0 rows
- interactions_not_materialized_yet gw_reqs > 0 AND steps == 0 (STDM lag)
- abandoned_before_llm steps > 0, LLM_STEP == 0, gw_reqs == 0
- planner_ran_no_gateway_logs LLM_STEP > 0 with generation ids, gw_reqs == 0
- complete the happy path
Order matters the gateway-direct rule sits BEFORE abandoned_before_llm
because ``gw_req_count > 0`` is a stronger positive signal than
``steps_total > 0``. Verified here by including a case with both signals
disjointly (steps==0 on the gateway-direct path).
"""
from __future__ import annotations
import unittest
from . import _bootstrap # noqa: F401 — sys.path setup
from fetch_dc import _classify_session_shape # type: ignore
_CASES = [
# (label, kwargs, expected)
(
"session_not_found when sessions.json is empty",
dict(sessions_count=0, steps_total=0, llm_step_count=0,
steps_with_generation_id=0, gw_req_count=0),
"session_not_found",
),
(
"session_not_found wins even when gw_reqs > 0 (sessions gate runs first)",
dict(sessions_count=0, steps_total=0, llm_step_count=0,
steps_with_generation_id=0, gw_req_count=5),
"session_not_found",
),
(
"interactions_not_materialized_yet — fresh session, gateway populated, STDM lagging",
dict(sessions_count=1, steps_total=0, llm_step_count=0,
steps_with_generation_id=0, gw_req_count=3),
"interactions_not_materialized_yet",
),
(
"abandoned_before_llm — steps created but no LLM step, no gateway calls",
dict(sessions_count=1, steps_total=2, llm_step_count=0,
steps_with_generation_id=0, gw_req_count=0),
"abandoned_before_llm",
),
(
"planner_ran_no_gateway_logs — LLM steps + gen ids but gateway empty",
dict(sessions_count=1, steps_total=3, llm_step_count=2,
steps_with_generation_id=2, gw_req_count=0),
"planner_ran_no_gateway_logs",
),
(
"complete — the normal bucket",
dict(sessions_count=1, steps_total=5, llm_step_count=3,
steps_with_generation_id=3, gw_req_count=4),
"complete",
),
]
class ClassifySessionShapeTests(unittest.TestCase):
"""Parametric truth-table for the 5-way enum."""
def test_all_shapes(self):
for label, kwargs, expected in _CASES:
with self.subTest(label=label):
self.assertEqual(_classify_session_shape(**kwargs), expected)
def test_gateway_direct_precedes_abandoned(self):
"""Regression guard: the new rule must fire before abandoned_before_llm.
If someone reorders the checks, a session with gw_reqs > 0 AND
steps > 0 AND LLM_STEP == 0 (edge case happens when Step rows
land while Interaction parent rows are still lagging) could fall
through incorrectly. Today the rules' inputs are disjoint
(gateway-direct needs steps==0), so the guard case uses steps==0
to exercise the ordering directly.
"""
shape = _classify_session_shape(
sessions_count=1,
steps_total=0,
llm_step_count=0,
steps_with_generation_id=0,
gw_req_count=1,
)
self.assertEqual(shape, "interactions_not_materialized_yet")
if __name__ == "__main__":
unittest.main()