From 31d36f5210079926d9b9de13628f0e228fdc7753 Mon Sep 17 00:00:00 2001 From: Marzi Golbaz Date: Wed, 1 Jul 2026 16:21:30 -0700 Subject: [PATCH] feat: add agentforce-grid skill MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Adds the Agentforce Grid (AI Workbench) skill for spreadsheet-like AI operations in Salesforce — agent testing with utterances, batch prompt processing, AI-driven data enrichment, and evaluation columns. Covers the /services/data/v66.0/public/grid/ Connect API and 12 column types (AI, Agent, AgentTest, Evaluation, Formula, Object, DataModelObject, PromptTemplate, InvocableAction, Action, Reference, Text). Skill previously lived at git.soma.salesforce.com/tmcgrath/agentforce-grid-ai-skills and github.com/Agentforce-emu/agentforce-grid-skills. --- skills/agentforce-grid/SKILL.md | 298 +++++ .../references/api-endpoints.md | 1001 ++++++++++++++ .../references/column-configs.md | 1009 ++++++++++++++ .../references/evaluation-types.md | 595 +++++++++ .../references/use-case-patterns.md | 663 ++++++++++ .../references/workflow-patterns.md | 1161 +++++++++++++++++ 6 files changed, 4727 insertions(+) create mode 100644 skills/agentforce-grid/SKILL.md create mode 100644 skills/agentforce-grid/references/api-endpoints.md create mode 100644 skills/agentforce-grid/references/column-configs.md create mode 100644 skills/agentforce-grid/references/evaluation-types.md create mode 100644 skills/agentforce-grid/references/use-case-patterns.md create mode 100644 skills/agentforce-grid/references/workflow-patterns.md diff --git a/skills/agentforce-grid/SKILL.md b/skills/agentforce-grid/SKILL.md new file mode 100644 index 0000000..ca1ac7c --- /dev/null +++ b/skills/agentforce-grid/SKILL.md @@ -0,0 +1,298 @@ +--- +name: agentforce-grid +description: "Use for Agentforce Grid (AI Workbench) — the spreadsheet-like interface for AI operations in Salesforce. ALWAYS use this skill when the user wants to: test or evaluate an Agentforce agent with utterances; create workbooks, worksheets, or columns; enrich Salesforce records with AI-generated content; add AI columns (SINGLE_SELECT, PLAIN_TEXT), Formula columns, Object columns, Evaluation columns, or Reference columns; import CSV data; compare agent versions; debug column config errors; query SObjects or Data Cloud in a grid; run evaluations (coherence, topic assertion, response match); or use any mcp__grid-connect__ tools. Covers the Grid Connect API for all column CRUD, cell operations, agent testing pipelines, and batch processing. Skip for standalone Apex, Flows, dashboards, reports, validation rules, or Einstein Bots that don't involve Grid." +--- + +# Agentforce Grid Skill + +Agentforce Grid (AI Workbench) is a spreadsheet-like interface for AI operations in Salesforce. Worksheets contain typed columns that query data, run agents, generate AI content, and evaluate outputs. + +**Hierarchy:** Workbook > Worksheet > Columns > Rows > Cells + +> **Note:** The MCP tool surface is being consolidated from ~65 tools to ~15. Tool names below reflect the current state. If a tool is not found, check `get_supported_types` or `get_column_types` for alternatives. + +## Before You Act (Mandatory Checklist) + +Run this checklist before creating or suggesting any column: + +1. **Read current grid state** -- call `get_worksheet_data` to see all existing columns, their IDs, and cell data +2. **Check for duplicates** -- does a column with the same name or purpose already exist? If so, suggest editing it (`edit_ai_prompt`, `edit_column`) instead of creating a new one +3. **Verify all referenced column IDs exist** -- every `columnId` in `referenceAttributes`, `inputColumnReference`, or `referenceColumnReference` MUST come from the current grid state. NEVER hallucinate column IDs or reference columns that do not exist yet +4. **Confirm column type matches the task** -- use the decision table below +5. **Confirm response format matches the use case** -- SINGLE_SELECT for filtering/sorting, PLAIN_TEXT for free-form content +6. **Check dependency order** -- source columns must exist before processing columns that reference them +7. **Check available resources** -- call `get_prompt_templates` or `get_invocable_actions` before creating an AI column that could be handled by existing platform capabilities + +## Column Types + +| Type | `type` value | Use Case | +|------|-------------|----------| +| AI | `"AI"` | LLM text generation with custom prompts | +| Agent | `"Agent"` | Run agent conversations | +| AgentTest | `"AgentTest"` | Batch-test agent with utterances | +| Evaluation | `"Evaluation"` | Evaluate agent/prompt outputs (13 types) | +| Formula | `"Formula"` | Deterministic computed values | +| Object | `"Object"` | Query Salesforce SObjects | +| DataModelObject | `"DataModelObject"` | Query Data Cloud DMOs | +| PromptTemplate | `"PromptTemplate"` | Execute GenAI prompt templates | +| InvocableAction | `"InvocableAction"` | Execute Flows or Apex | +| Action | `"Action"` | Execute platform actions | +| Reference | `"Reference"` | Extract fields via JSON path | +| Text | `"Text"` | Static/editable text or CSV import | + +**Casing:** Server is case-insensitive. PascalCase and UPPER_CASE both work. API returns vary by context. + +For complete JSON configs: [Column Configs Reference](references/column-configs.md) + +## Choosing the Right Column Type + +| Need | Type | Format | Why | +|------|------|--------|-----| +| Categorize/filter/sort | AI | SINGLE_SELECT | Limited options enable filtering and scanning | +| Generate free-text (email, summary) | AI | PLAIN_TEXT | Open-ended content needs free-form output | +| Deterministic computation | Formula | N/A | No LLM needed, exact and reproducible | +| Extract field from JSON/Object | Reference | N/A | Zero LLM cost, exact extraction | +| Score/rate on a scale | AI | SINGLE_SELECT | e.g., High/Medium/Low for scannable output | +| Task already handled by a prompt template | PromptTemplate | N/A | Pre-built, tested, maintained by the org | +| Task already handled by a Flow | InvocableAction | N/A | Deterministic logic, no LLM cost | + +**Key principle:** If output is used for filtering, sorting, or downstream comparison, use SINGLE_SELECT. Free-text defeats scanability. If a platform capability (prompt template, Flow, list view) already does the job, prefer it over AI. + +## Role-Aware Suggestions + +Before suggesting columns, understand the user's role and intent. Ask if unclear. + +| Role | Common Needs | Pipeline Pattern | +|------|-------------|------------------| +| Sales Rep | Deal risk, competitive intel, account priority | Object(Opps) -> AI(risk, SINGLE_SELECT) | +| CSM | Customer health, check-in emails | Object(Accounts/Cases) -> AI(analysis) -> AI(email, PLAIN_TEXT) | +| RevOps | Pipeline quality, data hygiene flags | Object(Opps) -> AI(quality flag, SINGLE_SELECT: Clean/Minor Issues/Needs Fix) | +| Dev/QA | Agent testing, flow testing | Text(utterances) -> AgentTest -> Evaluation | +| Admin | Data enrichment, bulk updates | Object/ListView -> AI -> Action(RecordUpdate) | + +A RevOps user needs data quality flags (SINGLE_SELECT), not outreach emails. A CSM needs customer-facing outputs, not pipeline metrics. + +## Evaluation Types + +| Type | Needs Reference Column | Supported Inputs | +|------|----------------------|------------------| +| COHERENCE | No | Agent, AgentTest, PromptTemplate | +| CONCISENESS | No | Agent, AgentTest, PromptTemplate | +| FACTUALITY | No | Agent, AgentTest, PromptTemplate | +| INSTRUCTION_FOLLOWING | No | Agent, AgentTest, PromptTemplate | +| COMPLETENESS | No | Agent, AgentTest, PromptTemplate | +| RESPONSE_MATCH | **Yes** | Agent, AgentTest | +| TOPIC_ASSERTION | **Yes** | Agent, AgentTest | +| ACTION_ASSERTION | **Yes** | Agent, AgentTest | +| LATENCY_ASSERTION | No | Agent, AgentTest | +| BOT_RESPONSE_RATING | **Yes** | Agent, AgentTest | +| EXPRESSION_EVAL | No | Agent, AgentTest | +| CUSTOM_LLM_EVALUATION | **Yes** | Agent, AgentTest | +| TASK_RESOLUTION | No | Agent, AgentTest (conversation-level) | + +For complete evaluation guidance: [Evaluation Types Reference](references/evaluation-types.md) + +## Dependency Rules + +Column creation must follow the dependency DAG. A column CANNOT reference a column that does not yet exist or that depends on it. + +1. **Source data** (Text, Object, DataModelObject) -- no dependencies +2. **Processing** (AI, Agent, AgentTest, PromptTemplate, InvocableAction) -- depend on source +3. **Extraction** (Reference) -- depends on processing columns +4. **Assessment** (Evaluation) -- depends on Agent/AgentTest/PromptTemplate +5. **Formula** -- any level, but must only reference existing columns + +**Always create columns sequentially.** Each `add_column` returns the new column ID needed by subsequent columns. Parallel creation leads to missing IDs and unpredictable ordering. + +## Leverage Existing Resources + +Before creating an AI column, check if platform capabilities handle the task: + +1. `get_prompt_templates` -- use PromptTemplate column for pre-built prompts +2. `get_invocable_actions` -- use InvocableAction column for deterministic logic +3. `get_list_views` -- use list view SOQL in Object column's advancedMode +4. Existing columns -- use Reference column to extract data already present + +AI columns should be reserved for tasks requiring LLM reasoning. + +## Critical Config Rules + +**Nested config structure (REQUIRED for ALL column types):** +```json +{ + "name": "Column Name", + "type": "AI", + "config": { + "type": "AI", + "queryResponseFormat": {"type": "EACH_ROW"}, + "autoUpdate": true, + "config": { + "autoUpdate": true + } + } +} +``` + +Even Text columns need `type` inside config. An empty `config: {}` will fail. + +**queryResponseFormat:** Use `EACH_ROW` when adding a column to a worksheet that already has data. Use `WHOLE_COLUMN` with `splitByType: "OBJECT_PER_ROW"` only when importing new records (Object/DataModelObject). + +**modelConfig:** Use the model `name` for both `modelId` and `modelName`. Call `get_llm_models` to list available models. Default: `sfdc_ai__DefaultGPT4Omni`. + +**AI columns:** Require `mode: "llm"`, `responseFormat` with `options` array (empty `[]` for PLAIN_TEXT). + +**Evaluation columns:** Types requiring reference columns (RESPONSE_MATCH, TOPIC_ASSERTION, ACTION_ASSERTION, BOT_RESPONSE_RATING, CUSTOM_LLM_EVALUATION) must include `referenceColumnReference`. + +For complete JSON configs for all 12 types: [Column Configs Reference](references/column-configs.md) + +## Common Patterns + +### Agent Testing Pipeline +``` +Text(utterances) -> Text(expected) -> AgentTest -> Evaluation(RESPONSE_MATCH) -> Evaluation(COHERENCE) +``` +**Quick path:** Use `setup_agent_test` for one-call creation of the full pipeline. + +### Data Enrichment +``` +Object(Accounts) -> AI(summary, PLAIN_TEXT) -> AI(sentiment, SINGLE_SELECT) +``` + +### Flow/Apex Testing +``` +Text(inputs) -> InvocableAction(Flow) -> Reference(extract output field) +``` + +For complete step-by-step MCP tool call examples: [Use Case Patterns Reference](references/use-case-patterns.md) + +## SF CLI Setup & Authentication + +Before using the Grid API, authenticate to a Salesforce org using the SF CLI. + +```bash +sf --version # Check if installed +sf org login web --alias my-org # Authenticate (add --instance-url for specific org) +sf org display --target-org my-org --json # Get access token + instance URL +``` + +**Instance URL formats:** `https://sdbX.testX.pc-rnd.pc-aws.salesforce.com/` (internal), `https://mycompany.my.salesforce.com` (production), `https://mycompany--sandbox.sandbox.my.salesforce.com` (sandbox). + +**SF CLI does NOT accept lightning domains.** If a user provides one (e.g., `orgfarm-xxx.test1.lightning.pc-rnd.force.com`), ask for the instance URL instead. + +For full auth instructions: `sf org login web --help` + +## MCP Tool Reference + +All tools use prefix `mcp__grid-connect__`. Grouped by operation type. + +### Orchestration (Start Here) + +| Tool | What It Does | +|------|-------------| +| `apply_grid` | Create/update a full grid from YAML DSL -- most powerful orchestration tool | +| `setup_agent_test` | One-call agent test pipeline: workbook + worksheet + columns + evaluations | +| `create_workbook_with_worksheet` | Create workbook + worksheet in one call | +| `poll_worksheet_status` | Poll until processing completes | +| `get_worksheet_summary` | Compact status overview | +| `run_worksheet` | Execute worksheet with optional `runStrategy: "ColumnByColumn"` for sequential | + +### Read State + +| Tool | What It Does | +|------|-------------| +| `get_workbooks` | List all workbooks | +| `get_workbook` | Get workbook details | +| `get_worksheet` | Get worksheet metadata | +| `get_worksheet_data` | **Primary read tool** -- full data with all columns, rows, cells | +| `get_worksheet_data_generic` | Generic format variant | +| `get_column_data` | Get cell data for one column | + +### Create & Modify + +| Tool | What It Does | +|------|-------------| +| `create_workbook` / `create_worksheet` | Create resources | +| `add_column` | Add column (returns new column ID) | +| `edit_column` | Update config AND reprocess all cells | +| `save_column` | Update config WITHOUT reprocessing | +| `reprocess_column` / `reprocess` | Reprocess cells (column or worksheet) | +| `delete_column` / `delete_workbook` / `delete_worksheet` | Delete resources | +| `add_rows` / `delete_rows` | Manage rows | +| `update_cells` | Update specific cells (use `fullContent`, not `displayContent`) | +| `paste_data` | Paste data matrix into grid | +| `trigger_row_execution` | Run processing for specific rows or cells | +| `import_csv` | Import CSV to worksheet | + +### Typed Mutations (Prefer Over Raw edit_column) + +These auto-fetch current config, merge changes, and handle references correctly. + +| Tool | What It Does | +|------|-------------| +| `edit_ai_prompt` | Edit AI column prompt, model, or response format | +| `edit_agent_config` | Edit Agent/AgentTest config | +| `edit_prompt_template` | Edit PromptTemplate column | +| `change_model` | Switch LLM model on AI or PromptTemplate column | +| `add_evaluation` | Add evaluation with auto-wired references | +| `update_filters` | Update Object/DMO query filters | + +### Discovery + +| Tool | What It Does | +|------|-------------| +| `get_agents` | List agents (use `includeDrafts` for unpublished) | +| `get_agent_variables` | Get agent context variables | +| `get_sobjects` / `get_sobject_fields_display` / `get_sobject_fields_filter` | SObject discovery | +| `get_dataspaces` / `get_data_model_objects` / `get_data_model_object_fields` | Data Cloud discovery | +| `get_llm_models` | List available LLM models | +| `get_evaluation_types` | List evaluation types available in the org | +| `get_column_types` / `get_supported_types` | List column types | +| `get_formula_functions` / `get_formula_operators` | Formula building helpers | +| `get_invocable_actions` / `describe_invocable_action` | Discover Flows/Apex | +| `get_prompt_templates` / `get_prompt_template` | Discover prompt templates | +| `get_list_views` / `get_list_view_soql` | List view SOQL | + +### AI Generation + +| Tool | What It Does | +|------|-------------| +| `create_column_from_utterance` | Create column from natural language description | +| `generate_soql` | Natural language to SOQL | +| `generate_json_path` | AI-assisted JSON path generation | +| `generate_test_columns` | Generate test column configs | +| `generate_ia_input` | Generate invocable action input payload | +| `get_url` | Generate Lightning Experience URLs | + +## Tool Operation Distinctions + +| Tool | Behavior | When to Use | +|------|----------|-------------| +| `edit_column` | Updates config AND reprocesses | Changing prompt, model, references | +| `save_column` | Updates config, NO reprocess | Renaming, display-only changes | +| `reprocess_column` | Reprocesses with current config | Source data changed, retrying failures | + +## State Refresh + +Call `get_worksheet_data` after any mutation (add_column, paste_data, trigger_row_execution) to get updated IDs and statuses. Use `poll_worksheet_status` for long-running operations. + +## Error Patterns + +| Situation | What to Do | +|-----------|-----------| +| Column creation returns error but column exists | Verify with `get_worksheet_data` -- column may have been created despite the error | +| Cell processing failures | Use `reprocess_column` or `trigger_row_execution` with failed row IDs | +| Duplicate column name | API rejects with `DuplicateColumnName` -- check existing columns first | +| Config validation error | Fix config per error message and retry | + +## Known Limits + +- 100 test suites per org, 20 runs per suite, 10 concurrent runs per org +- AI column batches: 25 rows/batch, 4 parallel threads for evaluations + +## Reference Documentation + +- **[Column Configs](references/column-configs.md)** -- Complete JSON for all 12 column types +- **[Evaluation Types](references/evaluation-types.md)** -- All 13 evaluation types with guidance +- **[API Endpoints](references/api-endpoints.md)** -- Complete endpoint docs with examples +- **[Use Case Patterns](references/use-case-patterns.md)** -- Common workflows with MCP tool call examples +- **[Workflow Patterns](references/workflow-patterns.md)** -- User interaction models, conversation examples, CI/CD patterns diff --git a/skills/agentforce-grid/references/api-endpoints.md b/skills/agentforce-grid/references/api-endpoints.md new file mode 100644 index 0000000..a04852e --- /dev/null +++ b/skills/agentforce-grid/references/api-endpoints.md @@ -0,0 +1,1001 @@ +# API Endpoints Reference + +Complete documentation for Agentforce Grid public Connect API. + +## Base URL + +``` +/services/data/v66.0/public/grid +``` + +All endpoints are prefixed with this base URL. + +--- + +## Workbook Operations + +### List All Workbooks + +``` +GET /workbooks +``` + +**Response:** +```json +{ + "workbooks": [ + { + "id": "1W4xx0000004xxxx", + "name": "My Workbook" + } + ] +} +``` + +**Note:** This endpoint returns minimal workbook data. Use `GET /workbooks/{id}` to get the `aiWorksheetList`. + +### Create Workbook + +``` +POST /workbooks +``` + +**Request:** +```json +{ + "name": "Agent Test Suite" +} +``` + +**Response:** +```json +{ + "id": "1W4xx0000004xxxx", + "name": "Agent Test Suite" +} +``` + +### Get Workbook + +``` +GET /workbooks/{workbookId} +``` + +**Response:** +```json +{ + "id": "1W4xx0000004xxxx", + "name": "Agent Test Suite", + "aiWorksheetList": [ + { + "id": "1W1xx0000004xxxx", + "name": "Test Cases", + "workbookId": "1W4xx0000004xxxx" + } + ] +} +``` + +### Delete Workbook + +``` +DELETE /workbooks/{workbookId} +``` + +**Response:** 204 No Content + +--- + +## Worksheet Operations + +### Create Worksheet + +``` +POST /worksheets +``` + +**Request:** +```json +{ + "name": "Agent Test Cases", + "workbookId": "1W4xx0000004xxxx" +} +``` + +**Response:** +```json +{ + "autoUpdate": true, + "cells": [], + "columns": [], + "id": "1W1xx0000004xxxx", + "name": "Agent Test Cases", + "rows": [], + "workbookId": "1W4xx0000004xxxx" +} +``` + +### Get Worksheet Metadata + +``` +GET /worksheets/{worksheetId} +``` + +**Response:** +```json +{ + "id": "1W1xx0000004xxxx", + "name": "Agent Test Cases", + "workbookId": "1W4xx0000004xxxx", + "cells": [...], + "columns": [ + { + "id": "1W5xx0000004xxxx", + "name": "Test Utterances", + "type": "Text", + "status": "New", + "config": {}, // Note: GET responses may return empty config + "precedingColumnId": null, + "worksheetId": "1W1xx0000004xxxx" + } + ], + "rows": ["row-id-1", "row-id-2", "row-id-3"] // Array of row ID strings +} +``` + +### Get Worksheet Data (RECOMMENDED) + +``` +GET /worksheets/{worksheetId}/data +``` + +Returns complete worksheet data including all cells. + +**IMPORTANT:** This is the most reliable endpoint for reading worksheet state. `GET /worksheets/{id}` may return empty columns/rows/cells even when they exist. Always use `/data` to get the full state. + +**Response:** +```json +{ + "id": "1W1xx0000004xxxx", + "name": "Agent Test Cases", + "columns": [...], + "rows": [...], + "columnData": { + "column-id-1": [ + { + "id": "cell-id-1", + "worksheetColumnId": "column-id-1", + "worksheetRowId": "row-id-1", + "displayContent": "Hello, I need help", + "status": "Complete" + } + ] + } +} +``` + +### Get Worksheet Data (Generic Format) + +``` +GET /worksheets/{worksheetId}/data-generic +``` + +Returns data in a generic JSON format. + +### Update Worksheet + +``` +PUT /worksheets/{worksheetId} +``` + +**Request:** +```json +{ + "name": "Updated Worksheet Name" +} +``` + +### Delete Worksheet + +``` +DELETE /worksheets/{worksheetId} +``` + +**Response:** 204 No Content + +### Get Supported Column Types + +``` +GET /worksheets/{worksheetId}/supported-columns +``` + +Returns column types available for this worksheet. + +**Response:** +```json +{ + "columnTypes": [ + {"category": null, "description": null, "icon": null, "label": "AI", "name": "AI"}, + {"category": null, "description": null, "icon": null, "label": "Object", "name": "Object"}, + {"category": null, "description": null, "icon": null, "label": "Agent", "name": "Agent"}, + {"category": null, "description": null, "icon": null, "label": "AgentTest", "name": "AgentTest"} + ] +} +``` + +--- + +## Column Operations + +### Add Column to Worksheet + +``` +POST /worksheets/{worksheetId}/columns +``` + +**Request (Text Column) - MUST include nested config with `type`:** +```json +{ + "name": "Test Utterances", + "type": "Text", + "config": { + "type": "Text", + "autoUpdate": true, + "config": { + "autoUpdate": true + } + } +} +``` + +**CRITICAL:** An empty `"config": {}` will fail with a deserialization error. Always include the `type` field. + +**Request (AgentTest Column):** +```json +{ + "name": "Agent Output", + "type": "AgentTest", + "config": { + "type": "AgentTest", + "numberOfRows": 50, + "queryResponseFormat": {"type": "EACH_ROW"}, + "autoUpdate": true, + "config": { + "autoUpdate": true, + "agentId": "0Xxxx0000000001CAA", + "agentVersion": "0X9xx0000000001CAA", + "inputUtterance": { + "columnId": "utterance-col-id", + "columnName": "Test Utterances", + "columnType": "Text" + }, + "contextVariables": [] + } + } +} +``` + +**Response:** +```json +{ + "id": "1W5xx0000004xxxx", + "name": "Agent Output", + "worksheetId": "1W1xx0000004xxxx", + "type": "AgentTest", + "status": "New", + "config": {...}, + "precedingColumnId": "previous-col-id" +} +``` + +### Update Column + +**IMPORTANT:** Column operations use `/worksheets/{wsId}/columns/{colId}/...` path, NOT `/columns/{colId}/...`. + +``` +PUT /worksheets/{worksheetId}/columns/{columnId} +``` + +**Request:** +```json +{ + "name": "Updated Column Name", + "config": { + // updated configuration + } +} +``` + +### Delete Column + +``` +DELETE /worksheets/{worksheetId}/columns/{columnId} +``` + +**Response:** 204 No Content + +### Save Column (Without Processing) + +``` +POST /worksheets/{worksheetId}/columns/{columnId}/save +``` + +Saves column configuration without triggering cell processing. + +**Request:** +```json +{ + "type": "Text", + "name": "Column Name", + "config": {} +} +``` + +**Note:** The `type` field is required in the save request body. + +### Reprocess Column + +``` +POST /worksheets/{worksheetId}/columns/{columnId}/reprocess +``` + +Reprocesses all cells in the column. Request body: `{}` + +### Get Column Data + +``` +GET /worksheets/{worksheetId}/columns/{columnId}/data +``` + +Returns all cell data for the column. + +**Response:** +```json +{ + "cells": [ + { + "id": "cell-id-1", + "worksheetColumnId": "column-id", + "worksheetRowId": "row-id-1", + "displayContent": "Agent response text...", + "fullContent": {...}, + "status": "Complete", + "statusMessage": null + } + ] +} +``` + +--- + +## Row Operations + +### Add Rows + +``` +POST /worksheets/{worksheetId}/rows +``` + +**Request:** +```json +{ + "numberOfRows": 10, + "anchorRowId": "existing-row-id" +} +``` + +**Note:** `anchorRowId` is required when the worksheet already has rows. + +**Response:** +```json +{ + "rowIds": [], + "rowsAdded": 10, + "success": true +} +``` + +### Delete Rows + +``` +POST /worksheets/{worksheetId}/delete-rows +``` + +**Request:** +```json +{ + "rowIds": ["row-id-1", "row-id-2", "row-id-3"] +} +``` + +--- + +## Cell Operations + +### Update Cells + +``` +PUT /worksheets/{worksheetId}/cells +``` + +**IMPORTANT:** Cell updates use `fullContent` (an object), NOT `displayContent` (which is read-only). + +**Request:** +```json +{ + "cells": [ + { + "id": "cell-id-1", + "fullContent": {"text": "New cell value"} + } + ] +} +``` + +**Note:** The `displayContent` field is read-only and cannot be used for updates. Use `fullContent` with an appropriate object structure. For populating cells with text data, prefer the Paste endpoint instead. +``` + +### Paste Data + +``` +POST /worksheets/{worksheetId}/paste +``` + +Paste a matrix of data into the worksheet. **Uses `matrix` field (not `data`).** + +**Request:** +```json +{ + "startColumnId": "column-id", + "startRowId": "row-id", + "matrix": [ + [{"displayContent": "row1-col1"}, {"displayContent": "row1-col2"}], + [{"displayContent": "row2-col1"}, {"displayContent": "row2-col2"}] + ] +} +``` + +### Trigger Row Execution + +``` +POST /worksheets/{worksheetId}/trigger-row-execution +``` + +Triggers processing for specified rows. + +**Request:** +```json +{ + "trigger": "RUN_ROW", + "rowIds": ["row-id-1", "row-id-2"] +} +``` + +**Trigger Types:** +- `RUN_ROW` - Process all columns for specified rows (most common) +- `RUN_SELECTION` - Process specific cells (use `seedCellIds` instead of `rowIds`) +- `EDIT` - Re-trigger after cell edit (use `editedCells` array) +- `PASTE` - Re-trigger after paste (use `startColumnId` and `matrix`) + +--- + +## Run Worksheet + +### Run Worksheet + +``` +POST /run-worksheet +``` + +Runs a worksheet with specific row inputs and column configuration. Returns a job ID for polling via `GET /run-worksheet/{jobId}`. + +**Request (default -- all columns in parallel):** +```json +{ + "worksheetId": "1W1xx0000004xxxx", + "rowInputs": [...], + "columnConfig": {...} +} +``` + +**Request (sequential -- columns run one at a time):** +```json +{ + "worksheetId": "1W1xx0000004xxxx", + "rowInputs": [...], + "columnConfig": {...}, + "runStrategy": "ColumnByColumn" +} +``` + +**Parameters:** + +| Field | Type | Required | Description | +|-------|------|----------|-------------| +| `worksheetId` | string | Yes | The worksheet to run | +| `rowInputs` | array | Yes | Row input data | +| `columnConfig` | object | Yes | Column configuration | +| `runStrategy` | string | No | Execution strategy. Omit (default) to run all columns in parallel. Set to `"ColumnByColumn"` to run columns sequentially, one at a time. | + +**Response:** +```json +{ + "jobId": "job-id-string" +} +``` + +Use `GET /run-worksheet/{jobId}` to poll for results. + +--- + +## CSV Import + +### Import CSV to Worksheet + +``` +POST /worksheets/{worksheetId}/import-csv +``` + +Creates Text columns from uploaded CSV file. + +--- + +## Agent Operations + +### Get Available Agents + +``` +GET /agents +``` + +Returns list of active agents in the org. + +**Response:** +```json +{ + "agents": [ + { + "id": "0Xxxx0000000001CAA", + "name": "Support Agent", + "activeVersion": "0X9xx0000000001CAA" + } + ] +} +``` + +### Get Agent Variables + +``` +GET /agents/{activeVersionId}/variables +``` + +Returns context variables for an agent version. Use the `activeVersion` ID from the agents list (not the agent ID). + +**Response:** +```json +{ + "variables": [ + { + "name": "VerifiedCustomerId", + "dataType": "Text", + "description": null, + "label": null + } + ] +} + +--- + +## Prompt Template Operations + +### Get Available Prompt Templates + +``` +GET /prompt-templates +``` + +**Response:** +```json +{ + "templates": [ + { + "id": "Generate_Customer_Email", + "developerName": "Generate_Customer_Email", + "name": "Generate_Customer_Email" + } + ] +} +``` + +### Get Prompt Template by Name + +``` +GET /prompt-templates/{promptTemplateDevName} +``` + +Returns detailed template information including inputs. + +--- + +## SObject Operations + +### Get Available SObjects + +``` +GET /sobjects +``` + +Returns list of queryable SObjects. + +**Response:** +```json +{ + "sobjects": [ + { + "apiName": "Account", + "label": "Account", + "pluralLabel": "Accounts" + } + ] +} +``` + +### Get Fields for Display + +``` +POST /sobjects/fields-display +``` + +Returns fields suitable for display. + +**Request:** +```json +{ + "sobjectList": ["Account", "Contact"] +} +``` + +**Note:** Use `sobjectList` in request body (array), NOT `objectApiName` query parameter. + +### Get Fields for Filtering + +``` +POST /sobjects/fields-filter +``` + +Returns fields suitable for filtering. + +**Request:** +```json +{ + "sobjectList": ["Account"] +} +``` + +### Get Fields for Record Update + +``` +POST /sobjects/fields-record-update +``` + +Returns fields that can be updated. + +**Request:** +```json +{ + "sobjectList": ["Account"] +} +``` + +--- + +## Data Cloud Operations + +### Get Available Dataspaces + +``` +GET /dataspaces +``` + +**Response:** +```json +{ + "dataspaces": [ + { + "name": "default", + "label": "Default Dataspace" + } + ] +} +``` + +### Get Data Model Objects + +``` +GET /dataspaces/{dataspace}/data-model-objects +``` + +Returns DMOs in the specified dataspace. + +### Get DMO Fields + +``` +GET /dataspaces/{dataspace}/data-model-objects/{dmoName}/fields +``` + +Returns fields for a specific DMO. + +--- + +## Invocable Action Operations + +### Get Available Invocable Actions + +``` +GET /invocable-actions +``` + +Returns list of available invocable actions (Flows, Apex, etc.). + +### Describe Invocable Action + +``` +GET /invocable-actions/describe?actionType=FLOW&actionName=Create_Case +``` + +Returns detailed information about a specific action including inputs/outputs. + +### Generate Invocable Action Input + +``` +POST /worksheets/{worksheetId}/generate-ia-input +``` + +Generates input payload for an invocable action. + +--- + +## Metadata Endpoints + +### Get Column Types + +``` +GET /column-types +``` + +Returns all available column types. + +### Get LLM Models + +``` +GET /llm-models +``` + +Returns available LLM models for AI columns. + +**Response:** +```json +{ + "models": [ + { + "name": "sfdc_ai__DefaultGPT4Omni", + "label": "GPT 4 Omni", + "maxContentLength": 16384, + "encodingType": null + }, + { + "name": "sfdc_ai__DefaultGPT5", + "label": "GPT 5", + "maxContentLength": 128000, + "encodingType": null + }, + { + "name": "sfdc_ai__DefaultBedrockAnthropicClaude45Sonnet", + "label": "Anthropic Claude Sonnet 4.5 on Amazon", + "maxContentLength": 8192, + "encodingType": null + } + ] +} +``` + +**Note:** The model `name` field is used for both `modelId` and `modelName` in column configs. Use `GET /llm-models` to discover all available models in your org. The API returns 37 models but not all are active. + +**Active models (16 total - no prefix in label):** + +**OpenAI:** +- `sfdc_ai__DefaultGPT41` (GPT 4.1) - 32768 tokens +- `sfdc_ai__DefaultGPT41Mini` (GPT 4.1 Mini) - 32768 tokens +- `sfdc_ai__DefaultGPT4Omni` (GPT 4 Omni) - 16384 tokens +- `sfdc_ai__DefaultGPT4OmniMini` (GPT 4 Omni Mini) - 16384 tokens +- `sfdc_ai__DefaultGPT5` (GPT 5) - 128000 tokens +- `sfdc_ai__DefaultGPT5Mini` (GPT 5 Mini) - 128000 tokens +- `sfdc_ai__DefaultO3` (O3) - 100000 tokens +- `sfdc_ai__DefaultO4Mini` (O4 Mini) - 100000 tokens +- `sfdc_ai__DefaultOpenAIGPT4OmniMini` (OpenAI GPT 4 Omni Mini) - 16384 tokens + +**Anthropic (via Amazon Bedrock):** +- `sfdc_ai__DefaultBedrockAnthropicClaude45Haiku` (Anthropic Claude Haiku 4.5 on Amazon) - 8192 tokens +- `sfdc_ai__DefaultBedrockAnthropicClaude45Sonnet` (Anthropic Claude Sonnet 4.5 on Amazon) - 8192 tokens +- `sfdc_ai__DefaultBedrockAnthropicClaude4Sonnet` (Anthropic Claude Sonnet 4 on Amazon) - 8192 tokens + +**Google (via Vertex AI):** +- `sfdc_ai__DefaultVertexAIGemini25Flash001` (Google Gemini 2.5 Flash) - 65536 tokens +- `sfdc_ai__DefaultVertexAIGemini25FlashLite001` (Google Gemini 2.5 Flash Lite) - 65536 tokens +- `sfdc_ai__DefaultVertexAIGeminiPro25` (Google Gemini 2.5 Pro) - 65536 tokens + +**Amazon:** +- `sfdc_ai__DefaultBedrockAmazonNovaLite` (Amazon Nova Lite) - 5000 tokens +- `sfdc_ai__DefaultBedrockAmazonNovaPro` (Amazon Nova Pro) - 5000 tokens + +**Model Status Indicators:** +- **No prefix** = Active and recommended +- **(Disabled)** = Beta or disabled models +- **(Rerouted)** = Legacy models redirected to newer versions +- **(Deprecated)** = Older versions being phased out + +### Get Evaluation Types + +``` +GET /evaluation-types +``` + +Returns available evaluation types. + +**Response:** +```json +{ + "types": [ + {"name": "RESPONSE_MATCH", "label": null, "description": null}, + {"name": "INSTRUCTION_FOLLOWING", "label": null, "description": null}, + {"name": "ACTION_ASSERTION", "label": null, "description": null}, + {"name": "TOPIC_ASSERTION", "label": null, "description": null}, + {"name": "CUSTOM_LLM_EVALUATION", "label": null, "description": null}, + {"name": "CONCISENESS", "label": null, "description": null}, + {"name": "EXPRESSION_EVAL", "label": null, "description": null}, + {"name": "FACTUALITY", "label": null, "description": null}, + {"name": "BOT_RESPONSE_RATING", "label": null, "description": null}, + {"name": "COMPLETENESS", "label": null, "description": null}, + {"name": "LATENCY_ASSERTION", "label": null, "description": null}, + {"name": "COHERENCE", "label": null, "description": null} + ] +} +``` + +**Note:** Returns `types` field (not `evaluationTypes`). + +### Get Formula Functions + +``` +GET /formula-functions +``` + +Returns available formula functions for Formula columns. + +### Get Formula Operators + +``` +GET /formula-operators +``` + +Returns available formula operators. + +### Get Supported Types + +``` +GET /supported-types +``` + +Returns all supported types in Agentforce Grid. + +--- + +## AI Generation Endpoints + +### Create Column from Utterance + +``` +POST /worksheets/{worksheetId}/create-column-from-utterance +``` + +Uses AI to create a column based on natural language description. + +**Request:** +```json +{ + "utterance": "Create an AI column that summarizes the account description" +} +``` + +### Generate SOQL from Natural Language + +``` +POST /generate-soql +``` + +Uses AI to generate SOQL from natural language. + +**Request:** +```json +{ + "text": "Get all accounts in the Technology industry" +} +``` + +**Response:** +```json +{ + "soql": "SELECT Name, Id FROM Account WHERE Industry = 'Technology' LIMIT 50" +} +``` + +**Note:** Uses the `text` field (not `utterance` or `objectApiName`). +``` + +### Generate JSON Path + +``` +POST /worksheets/{worksheetId}/generate-json-path +``` + +Uses AI to generate JSON path for extracting fields. + +--- + +## Formula Operations + +### Validate Formula + +``` +POST /worksheets/{worksheetId}/validate-formula +``` + +Validates a formula configuration. + +**Request:** +```json +{ + "formula": "CONCATENATE({$1}, ' ', {$2})", + "returnType": "string", + "referenceAttributes": [...] +} +``` + +--- + +## List View Operations + +### Get Available List Views + +``` +GET /list-views +``` + +Returns available list views. + +### Get List View SOQL + +``` +GET /list-views/{listViewId}/soql +``` + +Returns the SOQL query for a list view. + +--- + +## Error Handling + +All endpoints return standard error responses: + +```json +{ + "errorCode": "BAD_REQUEST", + "message": "Required parameter 'worksheetId' is missing" +} +``` + +**Common Error Codes:** +- `BAD_REQUEST` (400) - Invalid request parameters +- `UNAUTHORIZED` (401) - Authentication required +- `NOT_FOUND` (404) - Resource not found +- `INTERNAL_SERVER_ERROR` (500) - Server error diff --git a/skills/agentforce-grid/references/column-configs.md b/skills/agentforce-grid/references/column-configs.md new file mode 100644 index 0000000..6efe0b3 --- /dev/null +++ b/skills/agentforce-grid/references/column-configs.md @@ -0,0 +1,1009 @@ +# Column Configurations Reference + +Complete JSON configurations for all 12 Agentforce Grid column types. + +## CRITICAL: Config Structure + +**All column configs follow this nested structure:** + +```json +{ + "name": "Column Name", + "type": "AI", // Type value (see table below) + "config": { + "type": "AI", // type repeated inside config (REQUIRED) + "numberOfRows": 50, // Optional, at config level + "queryResponseFormat": {...}, // REQUIRED for processing columns + "autoUpdate": true, // REQUIRED + "config": { // Nested config with column-specific fields + "autoUpdate": true, + // ... column-specific configuration + } + } +} +``` + +## Type Values + +| Column Type | type value | columnType in referenceAttributes | +|-------------|------------|-----------------------------------| +| AI | `"Ai"` (canonical) or `"AI"` | `"AI"` | +| Object | `"Object"` | `"Object"` | +| Reference | `"Reference"` | `"Reference"` | +| Text | `"Text"` | `"Text"` | +| PromptTemplate | `"PromptTemplate"` | `"PromptTemplate"` | +| InvocableAction | `"InvocableAction"` | `"InvocableAction"` | +| Action | `"Action"` | `"Action"` | +| AgentTest | `"AgentTest"` | `"AgentTest"` | +| Agent | `"Agent"` | `"Agent"` | +| Evaluation | `"Evaluation"` | `"Evaluation"` | +| Formula | `"Formula"` | `"Formula"` | +| DataModelObject | `"DataModelObject"` | `"DataModelObject"` | + +**Note:** `type` and `columnType` in referenceAttributes both use PascalCase (e.g., `"Text"`, `"Object"`, `"AgentTest"`). The Connect API canonical wire value for AI columns is `"Ai"`, but the server is case-insensitive -- `"AI"`, `"Ai"`, `"ai"` all work. + +## queryResponseFormat (CRITICAL) + +**This determines whether a column imports new data or processes existing rows.** + +```json +// EACH_ROW - Process each existing row (USE THIS when worksheet already has data) +"queryResponseFormat": { + "type": "EACH_ROW" +} + +// WHOLE_COLUMN - Import new data into the worksheet +"queryResponseFormat": { + "type": "WHOLE_COLUMN", + "splitByType": "OBJECT_PER_ROW" +} +``` + +**IMPORTANT RULE**: When a worksheet already has data (from Object, Text, CSV import, etc.), new columns should use `EACH_ROW` to process each existing row. Only use `WHOLE_COLUMN` when importing new data. + +| Scenario | queryResponseFormat | +|----------|---------------------| +| Adding AI column to process existing Account data | `{"type": "EACH_ROW"}` | +| Adding Agent column to test with existing utterances | `{"type": "EACH_ROW"}` | +| Adding Object column to import new records | `{"type": "WHOLE_COLUMN", "splitByType": "OBJECT_PER_ROW"}` | +| Adding Text column from CSV import | `{"type": "WHOLE_COLUMN", "splitByType": "OBJECT_PER_ROW"}` | + +## ReferenceAttribute + +Used to reference other columns in configurations. **Use PascalCase for columnType.** + +```json +{ + "columnId": "1W5SB000005zk6H0AQ", + "columnName": "Accounts", + "columnType": "Object", + "fieldName": "Name" +} +``` + +**For Reference columns, fieldName can be empty:** +```json +{ + "columnId": "1W5SB0000060AqL0AU", + "columnName": "Leads.Title", + "columnType": "Reference" +} + +By default, `referenceAttributes` are optional (`isRequired` defaults to `false`). If the referenced cell is empty, an empty string is substituted. Set `"isRequired": true` on a referenceAttribute to block execution when the reference is missing -- the cell will get status `MissingInput` instead of processing. +``` + +## ModelConfig (Required for AI/PromptTemplate) + +**Always specify modelConfig. Use the model `name` for both fields:** + +```json +{ + "modelId": "sfdc_ai__DefaultGPT4Omni", + "modelName": "sfdc_ai__DefaultGPT4Omni" +} +``` + +**Recommended models (active, high-capability):** +- `sfdc_ai__DefaultGPT4Omni` - GPT 4 Omni (16K tokens) +- `sfdc_ai__DefaultGPT41` - GPT 4.1 (32K tokens) +- `sfdc_ai__DefaultGPT5` - GPT 5 (128K tokens) +- `sfdc_ai__DefaultGPT5Mini` - GPT 5 Mini (128K tokens) +- `sfdc_ai__DefaultO3` - O3 (100K tokens) +- `sfdc_ai__DefaultO4Mini` - O4 Mini (100K tokens) +- `sfdc_ai__DefaultBedrockAnthropicClaude45Sonnet` - Claude 4.5 Sonnet (8K tokens) +- `sfdc_ai__DefaultBedrockAnthropicClaude4Sonnet` - Claude 4 Sonnet (8K tokens) +- `sfdc_ai__DefaultVertexAIGemini25Flash001` - Gemini 2.5 Flash (64K tokens) +- `sfdc_ai__DefaultVertexAIGeminiPro25` - Gemini 2.5 Pro (64K tokens) + +Use `GET /llm-models` for the full list of available models in your org. + +--- + +## 1. AI Column + +Generate text using LLM with custom prompts. + +**Type value:** `"AI"` + +**Required fields in `config.config`:** +- `mode` (String, **required**) - Always `"llm"` +- `modelConfig` (ModelConfig, **required**) - LLM model selection +- `instruction` (String, **required**) - Prompt with `{$1}`, `{$2}` placeholders +- `referenceAttributes` (List, **required** when using placeholders) - Columns to substitute +- `responseFormat` (object, **required**) - Response format control + - `type`: `"PLAIN_TEXT"` or `"SINGLE_SELECT"` + - `options`: Array (empty `[]` for PLAIN_TEXT, or list of `{label, value}` for SINGLE_SELECT) + +**Example - Processing Existing Row Data (MOST COMMON):** +```json +{ + "name": "Draft Email", + "type": "AI", + "config": { + "type": "AI", + "numberOfRows": 20, + "queryResponseFormat": { + "type": "EACH_ROW" + }, + "autoUpdate": true, + "config": { + "autoUpdate": true, + "mode": "llm", + "modelConfig": { + "modelId": "sfdc_ai__DefaultGPT4Omni", + "modelName": "sfdc_ai__DefaultGPT4Omni" + }, + "instruction": "Write a personalized email for:\nName: {$1}\nTitle: {$2}\nCompany: {$3}", + "referenceAttributes": [ + {"columnId": "col-uuid-1", "columnName": "Leads", "columnType": "Object", "fieldName": "FirstName"}, + {"columnId": "col-uuid-2", "columnName": "Leads.Title", "columnType": "Reference"}, + {"columnId": "col-uuid-3", "columnName": "Leads.Company", "columnType": "Reference"} + ], + "responseFormat": { + "type": "PLAIN_TEXT", + "options": [] + } + } + } +} +``` + +**Example - Single Select Classification:** +```json +{ + "name": "Sentiment", + "type": "AI", + "config": { + "type": "AI", + "queryResponseFormat": { + "type": "EACH_ROW" + }, + "autoUpdate": true, + "config": { + "autoUpdate": true, + "mode": "llm", + "modelConfig": { + "modelId": "sfdc_ai__DefaultGPT4Omni", + "modelName": "sfdc_ai__DefaultGPT4Omni" + }, + "instruction": "Classify the sentiment of this text: {$1}", + "referenceAttributes": [ + {"columnId": "col-uuid-1", "columnName": "CustomerFeedback", "columnType": "Text"} + ], + "responseFormat": { + "type": "SINGLE_SELECT", + "options": [ + {"label": "Positive", "value": "positive"}, + {"label": "Negative", "value": "negative"}, + {"label": "Neutral", "value": "neutral"} + ] + } + } + } +} +``` + +--- + +## 2. Agent Column + +Run agent conversations with dynamic inputs. + +**Type value:** `"Agent"` + +**Required fields in `config.config`:** +- `agentId` (String, required) - Agent definition ID +- `agentVersion` (String, required) - Agent version ID +- `utterance` (String, required) - Message with `{$1}`, `{$2}` placeholders +- `utteranceReferences` (List, optional) - Columns for placeholder substitution +- `contextVariables` (List, optional) - Agent context variables +- `initialState` (ReferenceAttribute, optional) - For multi-turn conversations +- `conversationHistory` (ReferenceAttribute, optional) - For multi-turn + +**ContextVariable Structure:** +```json +// Static value +{ + "variableName": "Priority", + "type": "Text", + "value": "High" +} + +// Column reference +{ + "variableName": "CustomerName", + "type": "Text", + "reference": {"columnId": "col-uuid", "columnName": "Name", "columnType": "Object"} +} +``` + +**CRITICAL**: Each ContextVariable must have EITHER `value` OR `reference`, not both. + +**Note:** `testMode` is deprecated on Agent columns and has no effect. Do not include it in Agent column configs. + +**Example:** +```json +{ + "name": "Sales Agent Response", + "type": "Agent", + "config": { + "type": "Agent", + "numberOfRows": 50, + "queryResponseFormat": { + "type": "EACH_ROW" + }, + "autoUpdate": true, + "config": { + "autoUpdate": true, + "agentId": "0XxRM000000xxxxx", + "agentVersion": "0XyRM000000xxxxx", + "utterance": "Hello, I need help with {$1}", + "utteranceReferences": [ + {"columnId": "col-uuid-1", "columnName": "CustomerQuery", "columnType": "Text"} + ], + "contextVariables": [ + { + "variableName": "CustomerName", + "type": "Text", + "reference": {"columnId": "col-uuid-2", "columnName": "Customers", "columnType": "Object", "fieldName": "Name"} + }, + { + "variableName": "Priority", + "type": "Text", + "value": "High" + } + ] + } + } +} +``` + +--- + +## 3. AgentTest Column + +Test agents with utterances from a column. Used for batch testing. + +**Type value:** `"AgentTest"` + +**Required fields in `config.config`:** +- `agentId` (String, required) +- `agentVersion` (String, required) +- `inputUtterance` (ReferenceAttribute, required) - References Text column with test utterances +- `contextVariables` (List, optional) +- `initialState` (ReferenceAttribute, optional) +- `conversationHistory` (ReferenceAttribute, optional) +- `isDraft` (boolean, optional) - Test draft agent version +- `enableSimulationMode` (boolean, optional) + +**Example:** +```json +{ + "name": "Agent Test", + "type": "AgentTest", + "config": { + "type": "AgentTest", + "numberOfRows": 50, + "queryResponseFormat": { + "type": "EACH_ROW" + }, + "autoUpdate": true, + "config": { + "autoUpdate": true, + "agentId": "0XxRM000000xxxxx", + "agentVersion": "0XyRM000000xxxxx", + "inputUtterance": { + "columnId": "col-uuid-1", + "columnName": "Test Utterances", + "columnType": "Text" + }, + "contextVariables": [], + "isDraft": false, + "enableSimulationMode": false + } + } +} +``` + +**Example with Context Variables:** +```json +{ + "name": "Agent Test with Context", + "type": "AgentTest", + "config": { + "type": "AgentTest", + "numberOfRows": 50, + "queryResponseFormat": { + "type": "EACH_ROW" + }, + "autoUpdate": true, + "config": { + "autoUpdate": true, + "agentId": "0XxRM000000xxxxx", + "agentVersion": "0XyRM000000xxxxx", + "inputUtterance": { + "columnId": "col-uuid-1", + "columnName": "Utterances", + "columnType": "Text" + }, + "contextVariables": [ + { + "variableName": "AccountId", + "type": "Text", + "reference": {"columnId": "col-uuid-2", "columnName": "Accounts", "columnType": "Object", "fieldName": "Id"} + } + ] + } + } +} +``` + +--- + +#### Voice Testing (Conversation Level only) + +AgentTest columns support voice testing when `evaluationMode` is `"TEXT_VOICE"`: +- `voiceMode`: `"LIVE"` | `"REPLAY"` | `"VOICE_CONV_MODE"` +- `selectedPersonas`: Array of persona types (e.g., `"DEFAULT"`, `"FRUSTRATED"`) +- `noiseType`: Background noise simulation (`"traffic"`, `"factory"`, `"airport"`, `"marketplace"`, etc.) +- `noiseVolume`: 0.0 to 1.0 +- `audioEffects`: Object with sub-configs for lowPassFilter, clipping, gaussianNoise, packetLoss, compression + +--- + +## 4. Object Column + +Query Salesforce SObjects. + +**Type value:** `"Object"` + +**Required fields in `config.config`:** +- `objectApiName` (String, required) - SObject API name (e.g., "Account", "Contact") +- `fields` (List, required) - Fields to query with `name` and `type` +- `filters` (List, optional) - Filter conditions +- `advancedMode` (object, optional) - For raw SOQL queries + - Uses `inputs.queryString` for the SOQL query + - **IMPORTANT**: Placeholders use `{ColumnName}` or `{ColumnName.FieldName}` format (NOT `{$1}`) + +**FieldConfig Structure:** + +Fields must include both `name` and `type` properties. The `type` must match Salesforce field data types (UPPERCASE): + +```json +{"name": "Id", "type": "ID"} +{"name": "Name", "type": "STRING"} +{"name": "Industry", "type": "PICKLIST"} +{"name": "AnnualRevenue", "type": "CURRENCY"} +{"name": "Phone", "type": "PHONE"} +{"name": "Website", "type": "URL"} +{"name": "Description", "type": "TEXTAREA"} +``` + +**Common Field Types:** +- `ID`, `STRING`, `TEXTAREA` +- `INTEGER`, `DOUBLE`, `LONG`, `CURRENCY`, `PERCENT` +- `BOOLEAN`, `DATE`, `DATETIME`, `TIME` +- `PICKLIST`, `MULTIPICKLIST` +- `PHONE`, `EMAIL`, `URL` +- `REFERENCE`, `ADDRESS`, `LOCATION` + +Use `get_sobject_fields_display` or `get_sobject_fields_filter` tools to get the correct field types for your object. + +**FilterOperator Values:** +- `In`, `NotIn` +- `EqualTo`, `NotEqualTo` +- `Contains`, `StartsWith`, `EndsWith` +- `IsNull`, `IsNotNull` +- `LessThan`, `LessThanOrEqualTo`, `GreaterThan`, `GreaterThanOrEqualTo` + +**Example - Basic Query:** +```json +{ + "name": "Accounts", + "type": "Object", + "config": { + "type": "Object", + "numberOfRows": 50, + "queryResponseFormat": { + "type": "WHOLE_COLUMN", + "splitByType": "OBJECT_PER_ROW" + }, + "autoUpdate": true, + "config": { + "autoUpdate": true, + "objectApiName": "Account", + "fields": [ + {"name": "Id", "type": "ID"}, + {"name": "Name", "type": "STRING"}, + {"name": "Industry", "type": "PICKLIST"}, + {"name": "Description", "type": "TEXTAREA"} + ], + "filters": [ + { + "field": "Industry", + "operator": "In", + "values": [ + {"value": "Technology", "type": "STRING"}, + {"value": "Finance", "type": "STRING"} + ] + } + ] + } + } +} +``` + +**Example - Advanced SOQL:** +```json +{ + "name": "Custom Query", + "type": "Object", + "config": { + "type": "Object", + "numberOfRows": 50, + "queryResponseFormat": { + "type": "WHOLE_COLUMN", + "splitByType": "OBJECT_PER_ROW" + }, + "autoUpdate": true, + "config": { + "autoUpdate": true, + "objectApiName": "Account", + "advancedMode": { + "type": "SOQL", + "inputs": { + "queryString": "SELECT Id, Name, Industry FROM Account WHERE Industry = '{TargetIndustry}' AND CreatedDate > LAST_N_DAYS:30 LIMIT 50" + }, + "referenceAttributes": [ + {"columnId": "col-uuid-1", "columnName": "TargetIndustry", "columnType": "Text"} + ] + } + } + } +} +``` + +--- + +## 5. Formula Column + +Compute values using formula expressions. + +**Type value:** `"Formula"` + +**Required fields in `config.config`:** +- `formula` (String, required) - Formula with `{$1}`, `{$2}` placeholders +- `returnType` (String, required) - Return type +- `referenceAttributes` (List, required) - Columns referenced in formula + +**Common Return Types:** +- `string`, `boolean`, `double`, `integer`, `long` +- `date`, `datetime`, `time` +- `id`, `reference` + +**Example - String Concatenation:** +```json +{ + "name": "Full Name", + "type": "Formula", + "config": { + "type": "Formula", + "queryResponseFormat": { + "type": "EACH_ROW" + }, + "autoUpdate": true, + "config": { + "autoUpdate": true, + "formula": "CONCATENATE({$1}, ' ', {$2})", + "returnType": "string", + "referenceAttributes": [ + {"columnId": "col-uuid-1", "columnName": "FirstName", "columnType": "Text"}, + {"columnId": "col-uuid-2", "columnName": "LastName", "columnType": "Text"} + ] + } + } +} +``` + +**Example - Boolean Expression:** +```json +{ + "name": "Is High Value", + "type": "Formula", + "config": { + "type": "Formula", + "queryResponseFormat": { + "type": "EACH_ROW" + }, + "autoUpdate": true, + "config": { + "autoUpdate": true, + "formula": "{$1} > 100000", + "returnType": "boolean", + "referenceAttributes": [ + {"columnId": "col-uuid-1", "columnName": "Accounts", "columnType": "Object", "fieldName": "AnnualRevenue"} + ] + } + } +} +``` + +--- + +## 6. PromptTemplate Column + +Execute GenAI prompt templates. + +**Type value:** `"PromptTemplate"` + +**Required fields in `config.config`:** +- `promptTemplateDevName` (String, required) - Template developer name +- `promptTemplateVersionId` (String, optional) +- `promptTemplateType` (String, optional) - e.g., "flex" +- `modelConfig` (ModelConfig, required) - LLM model selection +- `promptTemplateInputConfigs` (List, required) - Input mappings + +**PromptTemplateInputConfig Structure:** +```json +{ + "referenceName": "InputVariableName", + "definition": "Description of the input", + "referenceAttribute": {"columnId": "...", "columnName": "...", "columnType": "..."} +} +``` + +**Example:** +```json +{ + "name": "Email Generator", + "type": "PromptTemplate", + "config": { + "type": "PromptTemplate", + "numberOfRows": 50, + "queryResponseFormat": { + "type": "EACH_ROW" + }, + "autoUpdate": true, + "config": { + "autoUpdate": true, + "promptTemplateDevName": "Generate_Customer_Email", + "promptTemplateType": "flex", + "modelConfig": { + "modelId": "sfdc_ai__DefaultGPT4Omni", + "modelName": "sfdc_ai__DefaultGPT4Omni" + }, + "promptTemplateInputConfigs": [ + { + "referenceName": "CustomerName", + "definition": "The customer's name", + "referenceAttribute": {"columnId": "col-uuid-1", "columnName": "Customers", "columnType": "Object", "fieldName": "Name"} + }, + { + "referenceName": "Issue", + "definition": "The customer's issue", + "referenceAttribute": {"columnId": "col-uuid-2", "columnName": "CaseDescription", "columnType": "Text"} + } + ] + } + } +} +``` + +--- + +## 7. InvocableAction Column + +Execute Flows, Apex, or other invocable actions. + +**Type value:** `"InvocableAction"` + +**Required fields in `config.config`:** +- `actionInfo` (object, required) + - `actionType`: `FLOW`, `APEX`, etc. + - `actionName`: Flow/Apex API name + - `url`: Action URL + - `label`: Display label +- `inputPayload` (String, required) - JSON payload with `{$1}` placeholders +- `referenceAttributes` (List, required) - Columns for placeholder substitution + +**Example - Flow Execution:** +```json +{ + "name": "Create Case", + "type": "InvocableAction", + "config": { + "type": "InvocableAction", + "numberOfRows": 50, + "queryResponseFormat": { + "type": "EACH_ROW" + }, + "autoUpdate": true, + "config": { + "autoUpdate": true, + "actionInfo": { + "actionType": "FLOW", + "actionName": "Create_Support_Case", + "url": "/services/data/v66.0/actions/custom/flow/Create_Support_Case", + "label": "Create Support Case" + }, + "inputPayload": "{\"Subject\": \"{$1}\", \"Description\": \"{$2}\", \"Priority\": \"{$3}\"}", + "referenceAttributes": [ + {"columnId": "col-uuid-1", "columnName": "Subject", "columnType": "Text"}, + {"columnId": "col-uuid-2", "columnName": "Description", "columnType": "Text"}, + {"columnId": "col-uuid-3", "columnName": "Priority", "columnType": "Text"} + ] + } + } +} +``` + +**Example - Apex Action:** +```json +{ + "name": "Process Record", + "type": "InvocableAction", + "config": { + "type": "InvocableAction", + "numberOfRows": 50, + "queryResponseFormat": { + "type": "EACH_ROW" + }, + "autoUpdate": true, + "config": { + "autoUpdate": true, + "actionInfo": { + "actionType": "APEX", + "actionName": "ProcessRecordAction", + "url": "/services/data/v66.0/actions/custom/apex/ProcessRecordAction", + "label": "Process Record" + }, + "inputPayload": "{\"recordId\": \"{$1}\"}", + "referenceAttributes": [ + {"columnId": "col-uuid-1", "columnName": "Records", "columnType": "Object", "fieldName": "Id"} + ] + } + } +} +``` + +--- + +## 8. Evaluation Column + +Evaluate agent or prompt outputs using built-in or custom evaluations. + +**Type value:** `"Evaluation"` + +**Required fields in `config.config`:** +- `evaluationType` (String, required) - One of 13 evaluation types +- `inputColumnReference` (ReferenceAttribute, required) - Column to evaluate +- `referenceColumnReference` (ReferenceAttribute, conditional) - For comparison evaluations +- `autoEvaluate` (boolean, optional) - Auto-run evaluation +- `expressionFormula` (String, optional) - For EXPRESSION_EVAL type +- `customEvaluation` (object, optional) - For CUSTOM_LLM_EVALUATION + +See [Evaluation Types Reference](evaluation-types.md) for complete guidance. + +**Example - Response Match:** +```json +{ + "name": "Response Match", + "type": "Evaluation", + "config": { + "type": "Evaluation", + "queryResponseFormat": { + "type": "EACH_ROW" + }, + "autoUpdate": true, + "config": { + "autoUpdate": true, + "evaluationType": "RESPONSE_MATCH", + "inputColumnReference": { + "columnId": "col-uuid-1", + "columnName": "Agent Output", + "columnType": "AgentTest" + }, + "referenceColumnReference": { + "columnId": "col-uuid-2", + "columnName": "Expected Response", + "columnType": "Text" + }, + "autoEvaluate": true + } + } +} +``` + +**Example - Quality Score (No Reference):** +```json +{ + "name": "Coherence Score", + "type": "Evaluation", + "config": { + "type": "Evaluation", + "queryResponseFormat": { + "type": "EACH_ROW" + }, + "autoUpdate": true, + "config": { + "autoUpdate": true, + "evaluationType": "COHERENCE", + "inputColumnReference": { + "columnId": "col-uuid-1", + "columnName": "Agent Output", + "columnType": "AgentTest" + }, + "autoEvaluate": true + } + } +} +``` + +--- + +## 9. DataModelObject Column + +Query Data Cloud Data Model Objects (DMOs). + +**Type value:** `"DataModelObject"` + +**Required fields in `config.config`:** +- `dataModelObjectApiName` (String, required) - DMO API name +- `dataspaceName` (String, required) - Data Cloud dataspace +- `fields` (List, required) - Fields to query with `name` and `type` +- `filters` (List, optional) - Filter conditions +- `advancedMode` (object, optional) - For DCSQL queries + - Uses `inputs.queryString` for the DCSQL query + - **IMPORTANT**: Placeholders use `{ColumnName}` or `{ColumnName.FieldName}` format (NOT `{$1}`) + +**Example:** +```json +{ + "name": "Unified Individuals", + "type": "DataModelObject", + "config": { + "type": "DataModelObject", + "numberOfRows": 50, + "queryResponseFormat": { + "type": "WHOLE_COLUMN", + "splitByType": "OBJECT_PER_ROW" + }, + "autoUpdate": true, + "config": { + "autoUpdate": true, + "dataModelObjectApiName": "UnifiedIndividual__dlm", + "dataspaceName": "default", + "fields": [ + {"name": "Id__c", "type": "string"}, + {"name": "FirstName__c", "type": "string"}, + {"name": "LastName__c", "type": "string"}, + {"name": "Email__c", "type": "string"} + ] + } + } +} +``` + +**Example - Advanced DCSQL:** +```json +{ + "name": "Custom DMO Query", + "type": "DataModelObject", + "config": { + "type": "DataModelObject", + "numberOfRows": 50, + "queryResponseFormat": { + "type": "WHOLE_COLUMN", + "splitByType": "OBJECT_PER_ROW" + }, + "autoUpdate": true, + "config": { + "autoUpdate": true, + "dataModelObjectApiName": "UnifiedIndividual__dlm", + "dataspaceName": "default", + "advancedMode": { + "type": "DCSQL", + "inputs": { + "queryString": "SELECT Id__c, FirstName__c, Email__c FROM UnifiedIndividual__dlm WHERE Email__c LIKE '%{EmailDomain}%'" + }, + "referenceAttributes": [ + {"columnId": "col-uuid-1", "columnName": "EmailDomain", "columnType": "Text"} + ] + } + } + } +} +``` + +--- + +## 10. Reference Column + +Extract specific fields from other columns using JSON path. + +**Type value:** `"Reference"` + +**Required fields in `config.config`:** +- `referenceColumnId` (String, required) - Source column ID +- `referenceField` (String, required) - JSON path to extract + +**Example - Extract Agent Topic:** +```json +{ + "name": "Agent Topic", + "type": "Reference", + "config": { + "type": "Reference", + "queryResponseFormat": { + "type": "EACH_ROW" + }, + "autoUpdate": true, + "config": { + "autoUpdate": true, + "referenceColumnId": "agent-test-col-uuid", + "referenceField": "response.topicName" + } + } +} +``` + +**Example - Extract Object Field (e.g., Account Name from Object column):** +```json +{ + "name": "Account Name", + "type": "Reference", + "config": { + "type": "Reference", + "queryResponseFormat": { + "type": "EACH_ROW" + }, + "autoUpdate": true, + "config": { + "autoUpdate": true, + "referenceColumnId": "accounts-col-uuid", + "referenceField": "Name" + } + } +} +``` + +**Example - Extract Nested Field:** +```json +{ + "name": "First Bot Message", + "type": "Reference", + "config": { + "type": "Reference", + "queryResponseFormat": { + "type": "EACH_ROW" + }, + "autoUpdate": true, + "config": { + "autoUpdate": true, + "referenceColumnId": "agent-col-uuid", + "referenceField": "response.botMessages[0].text" + } + } +} +``` + +--- + +## 11. Text Column + +Static or editable text input. Also supports CSV import. + +**Type value:** `"Text"` + +**CRITICAL:** Text columns CANNOT use an empty `config: {}`. The `type` field is required inside config. + +**Optional fields in `config.config`:** +- `documentId` (String, optional) - For CSV import +- `documentColumnIndex` (Integer, optional) - CSV column index +- `includeHeaders` (Boolean, optional) - Include CSV headers + +**Example - Manual Entry (Minimum required config):** +```json +{ + "name": "Test Utterances", + "type": "Text", + "config": { + "type": "Text", + "autoUpdate": true, + "config": { + "autoUpdate": true + } + } +} +``` + +**Example - CSV Import:** +```json +{ + "name": "Imported Data", + "type": "Text", + "config": { + "type": "Text", + "numberOfRows": 100, + "queryResponseFormat": { + "type": "WHOLE_COLUMN", + "splitByType": "OBJECT_PER_ROW" + }, + "autoUpdate": true, + "config": { + "autoUpdate": true, + "documentId": "069xxxxxxxxxxxxxxx", + "documentColumnIndex": 0, + "includeHeaders": true + } + } +} +``` + +--- + +## 12. Action Column + +Execute standard platform actions. + +**Type value:** `"Action"` + +Similar to InvocableAction but for standard Salesforce actions. + +**Required fields in `config.config`:** +- `actionName` (String, required) - Action API name +- `inputParams` (List, optional) - Input parameters with references + +**InputParam Structure:** +```json +{ + "inputParamName": "parameterName", + "referenceAttribute": {"columnId": "...", "columnName": "...", "columnType": "..."} +} +``` + +**Example:** +```json +{ + "name": "Chatter Post", + "type": "Action", + "config": { + "type": "Action", + "numberOfRows": 50, + "queryResponseFormat": { + "type": "EACH_ROW" + }, + "autoUpdate": true, + "config": { + "autoUpdate": true, + "actionName": "chatterPost", + "inputParams": [ + { + "inputParamName": "text", + "referenceAttribute": {"columnId": "col-uuid-1", "columnName": "Message", "columnType": "Text"} + } + ] + } + } +} +``` diff --git a/skills/agentforce-grid/references/evaluation-types.md b/skills/agentforce-grid/references/evaluation-types.md new file mode 100644 index 0000000..967185d --- /dev/null +++ b/skills/agentforce-grid/references/evaluation-types.md @@ -0,0 +1,595 @@ +# Evaluation Types Reference + +Complete guide to all 13 evaluation types in Agentforce Grid. + +## Overview + +Evaluation columns assess the quality or correctness of agent/prompt outputs. Some evaluations compare against expected values (requiring a reference column), while others assess quality independently. + +## Evaluation Types Summary + +| Type | Requires Reference | Supported Inputs | Use Case | +|------|-------------------|------------------|----------| +| `COHERENCE` | No | Agent, AgentTest, PromptTemplate | Assess logical flow | +| `CONCISENESS` | No | Agent, AgentTest, PromptTemplate | Assess brevity | +| `FACTUALITY` | No | Agent, AgentTest, PromptTemplate | Assess factual accuracy | +| `INSTRUCTION_FOLLOWING` | No | Agent, AgentTest, PromptTemplate | Assess instruction adherence | +| `COMPLETENESS` | No | Agent, AgentTest, PromptTemplate | Assess coverage | +| `RESPONSE_MATCH` | **Yes** | Agent, AgentTest | Compare to expected | +| `TOPIC_ASSERTION` (UI: "Subagent") | **Yes** | Agent, AgentTest | Verify topic routing | +| `ACTION_ASSERTION` | **Yes** | Agent, AgentTest | Verify actions | +| `LATENCY_ASSERTION` | No | Agent, AgentTest | Check response time | +| `BOT_RESPONSE_RATING` | **Yes** | Agent, AgentTest | Overall quality rating | +| `EXPRESSION_EVAL` | No | Agent, AgentTest | Custom formula | +| `CUSTOM_LLM_EVALUATION` | **Yes** | Agent, AgentTest | Custom LLM judge | +| `TASK_RESOLUTION` | No | Agent, AgentTest (conversation-level only) | Assess task completion | + +--- + +## Quality Assessment Evaluations (No Reference Required) + +These evaluations assess intrinsic quality without comparing to expected output. + +### COHERENCE + +Measures logical flow and consistency of the response. + +**Assesses:** +- Logical flow of ideas +- Consistency in reasoning +- Clear connection between statements + +**Example:** +```json +{ + "name": "Coherence Score", + "type": "Evaluation", + "config": { + "type": "Evaluation", + "queryResponseFormat": {"type": "EACH_ROW"}, + "autoUpdate": true, + "config": { + "autoUpdate": true, + "evaluationType": "COHERENCE", + "inputColumnReference": { + "columnId": "agent-output-col-id", + "columnName": "Agent Output", + "columnType": "AgentTest" + }, + "autoEvaluate": true + } + } +} +``` + +### CONCISENESS + +Measures brevity without losing important information. + +**Assesses:** +- Brevity without losing important information +- Elimination of redundancy +- Efficient communication + +**Example:** +```json +{ + "name": "Conciseness Score", + "type": "Evaluation", + "config": { + "type": "Evaluation", + "queryResponseFormat": {"type": "EACH_ROW"}, + "autoUpdate": true, + "config": { + "autoUpdate": true, + "evaluationType": "CONCISENESS", + "inputColumnReference": { + "columnId": "agent-output-col-id", + "columnName": "Agent Output", + "columnType": "AgentTest" + }, + "autoEvaluate": true + } + } +} +``` + +### FACTUALITY + +Measures factual accuracy of the response. + +**Assesses:** +- Correctness of stated facts +- Absence of misinformation +- Verifiable claims + +**Example:** +```json +{ + "name": "Factuality Score", + "type": "Evaluation", + "config": { + "type": "Evaluation", + "queryResponseFormat": {"type": "EACH_ROW"}, + "autoUpdate": true, + "config": { + "autoUpdate": true, + "evaluationType": "FACTUALITY", + "inputColumnReference": { + "columnId": "agent-output-col-id", + "columnName": "Agent Output", + "columnType": "AgentTest" + }, + "autoEvaluate": true + } + } +} +``` + +### INSTRUCTION_FOLLOWING + +Measures how well the response follows given instructions. + +**Assesses:** +- Adherence to specific requirements +- Compliance with format guidelines +- Following of step-by-step instructions + +**Example:** +```json +{ + "name": "Instruction Following", + "type": "Evaluation", + "config": { + "type": "Evaluation", + "queryResponseFormat": {"type": "EACH_ROW"}, + "autoUpdate": true, + "config": { + "autoUpdate": true, + "evaluationType": "INSTRUCTION_FOLLOWING", + "inputColumnReference": { + "columnId": "prompt-output-col-id", + "columnName": "Prompt Output", + "columnType": "PromptTemplate" + }, + "autoEvaluate": true + } + } +} +``` + +### COMPLETENESS + +Measures if the response fully addresses the query. + +**Assesses:** +- Coverage of all required information +- Depth of explanation +- Inclusion of necessary context + +**Example:** +```json +{ + "name": "Completeness Score", + "type": "Evaluation", + "config": { + "type": "Evaluation", + "queryResponseFormat": {"type": "EACH_ROW"}, + "autoUpdate": true, + "config": { + "autoUpdate": true, + "evaluationType": "COMPLETENESS", + "inputColumnReference": { + "columnId": "agent-output-col-id", + "columnName": "Agent Output", + "columnType": "Agent" + }, + "autoEvaluate": true + } + } +} +``` + +--- + +## Comparison Evaluations (Reference Required) + +These evaluations compare output against expected values. + +### RESPONSE_MATCH + +Compares agent's response to an expected response. + +**Assesses:** +- Content similarity and accuracy +- Tone and style consistency +- Completeness of information + +**Required:** `referenceColumnReference` pointing to expected response column + +**Example:** +```json +{ + "name": "Response Match", + "type": "Evaluation", + "config": { + "type": "Evaluation", + "queryResponseFormat": {"type": "EACH_ROW"}, + "autoUpdate": true, + "config": { + "autoUpdate": true, + "evaluationType": "RESPONSE_MATCH", + "inputColumnReference": { + "columnId": "agent-output-col-id", + "columnName": "Agent Output", + "columnType": "AgentTest" + }, + "referenceColumnReference": { + "columnId": "expected-response-col-id", + "columnName": "Expected Response", + "columnType": "Text" + }, + "autoEvaluate": true + } + } +} +``` + +### TOPIC_ASSERTION + +**Note:** This evaluation type appears as "Subagent" in the Grid UI. The API name remains `topic_sequence_match`. + +Validates that the agent correctly identified and used the expected topic. + +**Assesses:** +- Topic selection accuracy +- Matches expected topic from reference column + +**Required:** `referenceColumnReference` pointing to expected topic column + +**Example:** +```json +{ + "name": "Topic Assertion", + "type": "Evaluation", + "config": { + "type": "Evaluation", + "queryResponseFormat": {"type": "EACH_ROW"}, + "autoUpdate": true, + "config": { + "autoUpdate": true, + "evaluationType": "TOPIC_ASSERTION", + "inputColumnReference": { + "columnId": "agent-output-col-id", + "columnName": "Agent Output", + "columnType": "AgentTest" + }, + "referenceColumnReference": { + "columnId": "expected-topic-col-id", + "columnName": "Expected Topic", + "columnType": "Text" + }, + "autoEvaluate": true + } + } +} +``` + +### ACTION_ASSERTION + +Validates that the agent executed the expected actions. + +**Assesses:** +- Action execution accuracy +- All expected actions were performed + +**Required:** `referenceColumnReference` pointing to expected actions column + +**Example:** +```json +{ + "name": "Action Assertion", + "type": "Evaluation", + "config": { + "type": "Evaluation", + "queryResponseFormat": {"type": "EACH_ROW"}, + "autoUpdate": true, + "config": { + "autoUpdate": true, + "evaluationType": "ACTION_ASSERTION", + "inputColumnReference": { + "columnId": "agent-output-col-id", + "columnName": "Agent Output", + "columnType": "AgentTest" + }, + "referenceColumnReference": { + "columnId": "expected-actions-col-id", + "columnName": "Expected Actions", + "columnType": "Text" + }, + "autoEvaluate": true + } + } +} +``` + +### BOT_RESPONSE_RATING + +Provides overall quality rating comparing to expected response. + +**Assesses:** +- Overall response quality +- Comparison against expected baseline + +**Required:** `referenceColumnReference` pointing to expected response column + +**Example:** +```json +{ + "name": "Response Rating", + "type": "Evaluation", + "config": { + "type": "Evaluation", + "queryResponseFormat": {"type": "EACH_ROW"}, + "autoUpdate": true, + "config": { + "autoUpdate": true, + "evaluationType": "BOT_RESPONSE_RATING", + "inputColumnReference": { + "columnId": "agent-output-col-id", + "columnName": "Agent Output", + "columnType": "AgentTest" + }, + "referenceColumnReference": { + "columnId": "expected-response-col-id", + "columnName": "Expected Response", + "columnType": "Text" + }, + "autoEvaluate": true + } + } +} +``` + +--- + +## Performance Evaluation + +### LATENCY_ASSERTION + +Validates that response time meets performance requirements. + +**Assesses:** +- Response time is within acceptable limits + +**Example:** +```json +{ + "name": "Latency Check", + "type": "Evaluation", + "config": { + "type": "Evaluation", + "queryResponseFormat": {"type": "EACH_ROW"}, + "autoUpdate": true, + "config": { + "autoUpdate": true, + "evaluationType": "LATENCY_ASSERTION", + "inputColumnReference": { + "columnId": "agent-output-col-id", + "columnName": "Agent Output", + "columnType": "AgentTest" + }, + "autoEvaluate": true + } + } +} +``` + +--- + +## Custom Evaluations + +### EXPRESSION_EVAL + +Evaluate using a custom formula expression over the agent response JSON. + +**Fields:** +- `expressionFormula` (String, required) - Formula with JSON path references +- `expressionReturnType` (String, optional) - Expected return type + +**Example - Check Topic Name:** +```json +{ + "name": "Topic Check", + "type": "Evaluation", + "config": { + "type": "Evaluation", + "queryResponseFormat": {"type": "EACH_ROW"}, + "autoUpdate": true, + "config": { + "autoUpdate": true, + "evaluationType": "EXPRESSION_EVAL", + "inputColumnReference": { + "columnId": "agent-output-col-id", + "columnName": "Agent Output", + "columnType": "AgentTest" + }, + "expressionFormula": "{response.topicName} == 'Service'", + "expressionReturnType": "boolean", + "autoEvaluate": true + } + } +} +``` + +**Example - Check Message Count:** +```json +{ + "name": "Message Count Check", + "type": "Evaluation", + "config": { + "type": "Evaluation", + "queryResponseFormat": {"type": "EACH_ROW"}, + "autoUpdate": true, + "config": { + "autoUpdate": true, + "evaluationType": "EXPRESSION_EVAL", + "inputColumnReference": { + "columnId": "agent-output-col-id", + "columnName": "Agent Output", + "columnType": "AgentTest" + }, + "expressionFormula": "LEN({response.botMessages}) > 0", + "expressionReturnType": "boolean", + "autoEvaluate": true + } + } +} +``` + +### CUSTOM_LLM_EVALUATION + +Use a custom prompt template for LLM-as-a-Judge evaluation. + +**Fields:** +- `customEvaluation` (object, required) + - `type`: "llm" + - `apiName`: Prompt template API name + - `customInput`: Optional custom input + +**Required:** `referenceColumnReference` for comparison baseline + +**Example:** +```json +{ + "name": "Custom Quality Check", + "type": "Evaluation", + "config": { + "type": "Evaluation", + "queryResponseFormat": {"type": "EACH_ROW"}, + "autoUpdate": true, + "config": { + "autoUpdate": true, + "evaluationType": "CUSTOM_LLM_EVALUATION", + "inputColumnReference": { + "columnId": "agent-output-col-id", + "columnName": "Agent Output", + "columnType": "AgentTest" + }, + "referenceColumnReference": { + "columnId": "criteria-col-id", + "columnName": "Evaluation Criteria", + "columnType": "Text" + }, + "customEvaluation": { + "type": "llm", + "apiName": "Custom_Evaluation_Template" + }, + "autoEvaluate": true + } + } +} +``` + +--- + +### TASK_RESOLUTION + +Assesses whether the agent successfully resolved the user's task across the full conversation. + +**IMPORTANT:** Only works with conversation-level agent tests (`testType: "CONVERSATION_LEVEL"`). Does not apply to turn-level tests. + +**Assesses:** +- Whether the agent achieved the user's stated goal +- End-to-end task completion across multiple turns + +**Example:** +```json +{ + "name": "Task Resolution", + "type": "Evaluation", + "config": { + "type": "Evaluation", + "queryResponseFormat": {"type": "EACH_ROW"}, + "autoUpdate": true, + "config": { + "autoUpdate": true, + "evaluationType": "TASK_RESOLUTION", + "inputColumnReference": { + "columnId": "agent-output-col-id", + "columnName": "Agent Output", + "columnType": "AgentTest" + }, + "autoEvaluate": true + } + } +} +``` + +--- + +## Selection Guide + +### For Quality Assessment (No Expected Output) + +When you want to assess intrinsic quality without a reference: + +- **COHERENCE** - Is the response logically structured? +- **CONCISENESS** - Is the response appropriately brief? +- **FACTUALITY** - Are the facts accurate? +- **INSTRUCTION_FOLLOWING** - Did it follow instructions? +- **COMPLETENESS** - Did it cover everything needed? + +### For Comparing Against Expected Output + +When you have expected/golden responses: + +- **RESPONSE_MATCH** - Does the response match expected? +- **BOT_RESPONSE_RATING** - Overall quality vs expected + +### For Agent Routing Validation + +When testing agent topic/action routing: + +- **TOPIC_ASSERTION** - Did it route to correct topic? +- **ACTION_ASSERTION** - Did it execute correct actions? + +### For Performance Testing + +- **LATENCY_ASSERTION** - Is response time acceptable? + +### For Conversation-Level Testing + +- **TASK_RESOLUTION** - Did the agent resolve the task? (conversation-level only) + +### For Custom Logic + +- **EXPRESSION_EVAL** - Custom formula over response JSON +- **CUSTOM_LLM_EVALUATION** - Custom LLM judge with your criteria + +--- + +## Common Patterns + +### Agent Testing Suite + +``` +Text: "Utterances" +Text: "Expected Responses" +Text: "Expected Topics" +AgentTest: "Agent Output" +Evaluation: "Response Match" (RESPONSE_MATCH) +Evaluation: "Topic Check" (TOPIC_ASSERTION) +Evaluation: "Coherence" (COHERENCE) +Evaluation: "Latency" (LATENCY_ASSERTION) +``` + +### Prompt Quality Evaluation + +``` +Text: "Inputs" +PromptTemplate: "Output" +Evaluation: "Coherence" (COHERENCE) +Evaluation: "Completeness" (COMPLETENESS) +Evaluation: "Instruction Following" (INSTRUCTION_FOLLOWING) +``` diff --git a/skills/agentforce-grid/references/use-case-patterns.md b/skills/agentforce-grid/references/use-case-patterns.md new file mode 100644 index 0000000..86dad14 --- /dev/null +++ b/skills/agentforce-grid/references/use-case-patterns.md @@ -0,0 +1,663 @@ +# Use Case Patterns Reference + +Complete workflow examples for common Agentforce Grid use cases using MCP tool calls. + +All tool calls use the MCP server prefix `mcp__grid-connect-mcp__`. For brevity, examples show just the tool name and parameters. + +--- + +## Pattern 1: Agent Testing Pipeline + +**Goal:** Test an agent with different utterances and evaluate responses. + +### Column Setup + +| Order | Column Name | Type | Purpose | +|-------|-------------|------|---------| +| 1 | Test Utterances | Text | Input test cases | +| 2 | Expected Responses | Text | Ground truth (optional) | +| 3 | Expected Topics | Text | Expected topic routing (optional) | +| 4 | Agent Output | AgentTest | Run agent | +| 5 | Response Match | Evaluation | Compare to expected | +| 6 | Topic Check | Evaluation | Verify topic routing | +| 7 | Quality Score | Evaluation | Assess coherence | + +### Quick Path: setup_agent_test + +For the most common case, use the all-in-one orchestration tool: + +``` +setup_agent_test({ + agentId: "0XxRM000000xxxxx", + agentVersion: "0XyRM000000xxxxx", + utterances: [ + "I need help resetting my password", + "What's my account balance?", + "Transfer me to a human" + ], + workbookName: "Agent Test Suite", + worksheetName: "Sales Agent Tests", + evaluationTypes: ["COHERENCE", "RESPONSE_MATCH", "TOPIC_ASSERTION"], + expectedResponses: [ + "I can help you reset your password...", + "Your account balance is...", + "Let me transfer you..." + ] +}) +``` + +This single call creates the workbook, worksheet, Text columns, pastes data, adds the AgentTest column, and wires up evaluations. + +### Step-by-Step Implementation (Manual) + +Use this approach when you need more control over the pipeline. + +**Step 1: Create Workbook and Worksheet** + +``` +create_workbook_with_worksheet({ + workbookName: "Agent Test Suite", + worksheetName: "Sales Agent Tests" +}) +// Returns: { workbookId, worksheetId, ... } +``` + +**Step 2: Add Text Column for Utterances** + +``` +add_column({ + worksheetId: "{worksheetId}", + name: "Test Utterances", + type: "Text", + config: '{"name":"Test Utterances","type":"Text","config":{"type":"Text","autoUpdate":true,"config":{"autoUpdate":true}}}' +}) +``` + +**Step 3: Add Text Column for Expected Responses** + +``` +add_column({ + worksheetId: "{worksheetId}", + name: "Expected Responses", + type: "Text", + config: '{"name":"Expected Responses","type":"Text","config":{"type":"Text","autoUpdate":true,"config":{"autoUpdate":true}}}' +}) +``` + +**Step 4: Add Text Column for Expected Topics** + +``` +add_column({ + worksheetId: "{worksheetId}", + name: "Expected Topics", + type: "Text", + config: '{"name":"Expected Topics","type":"Text","config":{"type":"Text","autoUpdate":true,"config":{"autoUpdate":true}}}' +}) +``` + +**Step 5: Paste Test Data** + +``` +// First, get worksheet data to find row IDs +get_worksheet_data({ worksheetId: "{worksheetId}" }) + +// Paste utterances into the Text columns +paste_data({ + worksheetId: "{worksheetId}", + startColumnId: "{utterances-column-id}", + startRowId: "{first-row-id}", + matrix: '[[{"displayContent":"I need help resetting my password"}],[{"displayContent":"What is my account balance?"}]]' +}) +``` + +**Step 6: Add AgentTest Column** + +``` +add_column({ + worksheetId: "{worksheetId}", + name: "Agent Output", + type: "AgentTest", + config: '{"name":"Agent Output","type":"AgentTest","config":{"type":"AgentTest","numberOfRows":50,"queryResponseFormat":{"type":"EACH_ROW"},"autoUpdate":true,"config":{"autoUpdate":true,"agentId":"0XxRM000000xxxxx","agentVersion":"0XyRM000000xxxxx","inputUtterance":{"columnId":"{utterances-column-id}","columnName":"Test Utterances","columnType":"Text"},"contextVariables":[]}}}' +}) +``` + +**Step 7: Add Response Match Evaluation** + +``` +add_column({ + worksheetId: "{worksheetId}", + name: "Response Match", + type: "Evaluation", + config: '{"name":"Response Match","type":"Evaluation","config":{"type":"Evaluation","queryResponseFormat":{"type":"EACH_ROW"},"autoUpdate":true,"config":{"autoUpdate":true,"evaluationType":"RESPONSE_MATCH","inputColumnReference":{"columnId":"{agent-output-column-id}","columnName":"Agent Output","columnType":"AgentTest"},"referenceColumnReference":{"columnId":"{expected-responses-column-id}","columnName":"Expected Responses","columnType":"Text"},"autoEvaluate":true}}}' +}) +``` + +**Step 8: Add Topic Assertion Evaluation** + +``` +add_column({ + worksheetId: "{worksheetId}", + name: "Topic Check", + type: "Evaluation", + config: '{"name":"Topic Check","type":"Evaluation","config":{"type":"Evaluation","queryResponseFormat":{"type":"EACH_ROW"},"autoUpdate":true,"config":{"autoUpdate":true,"evaluationType":"TOPIC_ASSERTION","inputColumnReference":{"columnId":"{agent-output-column-id}","columnName":"Agent Output","columnType":"AgentTest"},"referenceColumnReference":{"columnId":"{expected-topics-column-id}","columnName":"Expected Topics","columnType":"Text"},"autoEvaluate":true}}}' +}) +``` + +**Step 9: Add Coherence Evaluation** + +``` +add_column({ + worksheetId: "{worksheetId}", + name: "Quality Score", + type: "Evaluation", + config: '{"name":"Quality Score","type":"Evaluation","config":{"type":"Evaluation","queryResponseFormat":{"type":"EACH_ROW"},"autoUpdate":true,"config":{"autoUpdate":true,"evaluationType":"COHERENCE","inputColumnReference":{"columnId":"{agent-output-column-id}","columnName":"Agent Output","columnType":"AgentTest"},"autoEvaluate":true}}}' +}) +``` + +**Step 10: Monitor Processing** + +``` +poll_worksheet_status({ + worksheetId: "{worksheetId}", + maxAttempts: 30, + intervalMs: 3000 +}) +``` + +Or for a one-time status check: + +``` +get_worksheet_summary({ worksheetId: "{worksheetId}" }) +``` + +--- + +## Pattern 2: Data Enrichment with AI + +**Goal:** Enrich Salesforce Account records with AI-generated summaries. + +### Column Setup + +| Order | Column Name | Type | Purpose | +|-------|-------------|------|---------| +| 1 | Accounts | Object | Query Account records | +| 2 | Company Summary | AI | Generate summaries | + +### Step-by-Step Implementation + +**Step 1: Create Worksheet** + +``` +create_workbook_with_worksheet({ + workbookName: "Account Enrichment", + worksheetName: "Tech Account Enrichment" +}) +``` + +**Step 2: Add Object Column** + +``` +add_column({ + worksheetId: "{worksheetId}", + name: "Accounts", + type: "Object", + config: '{"name":"Accounts","type":"Object","config":{"type":"Object","numberOfRows":50,"queryResponseFormat":{"type":"WHOLE_COLUMN","splitByType":"OBJECT_PER_ROW"},"autoUpdate":true,"config":{"autoUpdate":true,"objectApiName":"Account","fields":[{"name":"Id","type":"ID"},{"name":"Name","type":"STRING"},{"name":"Industry","type":"PICKLIST"},{"name":"Description","type":"TEXTAREA"},{"name":"AnnualRevenue","type":"CURRENCY"}],"filters":[{"field":"Industry","operator":"In","values":[{"value":"Technology","type":"STRING"},{"value":"Finance","type":"STRING"}]}]}}}' +}) +``` + +**Step 3: Add AI Column** + +``` +add_column({ + worksheetId: "{worksheetId}", + name: "Company Summary", + type: "AI", + config: '{"name":"Company Summary","type":"AI","config":{"type":"AI","numberOfRows":50,"queryResponseFormat":{"type":"EACH_ROW"},"autoUpdate":true,"config":{"autoUpdate":true,"mode":"llm","modelConfig":{"modelId":"sfdc_ai__DefaultGPT4Omni","modelName":"sfdc_ai__DefaultGPT4Omni"},"instruction":"Write a brief 2-3 sentence summary of this company based on the following information:\\n\\nCompany Name: {$1}\\nIndustry: {$2}\\nDescription: {$3}\\nAnnual Revenue: {$4}\\n\\nFocus on their market position and key business characteristics.","referenceAttributes":[{"columnId":"{accounts-column-id}","columnName":"Accounts","columnType":"Object","fieldName":"Name"},{"columnId":"{accounts-column-id}","columnName":"Accounts","columnType":"Object","fieldName":"Industry"},{"columnId":"{accounts-column-id}","columnName":"Accounts","columnType":"Object","fieldName":"Description"},{"columnId":"{accounts-column-id}","columnName":"Accounts","columnType":"Object","fieldName":"AnnualRevenue"}],"responseFormat":{"type":"PLAIN_TEXT","options":[]}}}}' +}) +``` + +**Step 4: Monitor** + +``` +poll_worksheet_status({ worksheetId: "{worksheetId}" }) +``` + +--- + +## Pattern 3: Prompt Template Batch Processing + +**Goal:** Run a prompt template across a dataset and evaluate quality. + +### Column Setup + +| Order | Column Name | Type | Purpose | +|-------|-------------|------|---------| +| 1 | Customer Names | Text | Input data | +| 2 | Issues | Text | Input data | +| 3 | Generated Emails | PromptTemplate | Execute template | +| 4 | Coherence | Evaluation | Quality check | +| 5 | Completeness | Evaluation | Coverage check | + +### Step-by-Step Implementation + +**Step 1: Create Input Columns** + +``` +create_workbook_with_worksheet({ + workbookName: "Prompt Testing", + worksheetName: "Email Generator" +}) +``` + +``` +add_column({ + worksheetId: "{worksheetId}", + name: "Customer Names", + type: "Text", + config: '{"name":"Customer Names","type":"Text","config":{"type":"Text","autoUpdate":true,"config":{"autoUpdate":true}}}' +}) +``` + +``` +add_column({ + worksheetId: "{worksheetId}", + name: "Issues", + type: "Text", + config: '{"name":"Issues","type":"Text","config":{"type":"Text","autoUpdate":true,"config":{"autoUpdate":true}}}' +}) +``` + +**Step 2: Add PromptTemplate Column** + +``` +add_column({ + worksheetId: "{worksheetId}", + name: "Generated Emails", + type: "PromptTemplate", + config: '{"name":"Generated Emails","type":"PromptTemplate","config":{"type":"PromptTemplate","numberOfRows":50,"queryResponseFormat":{"type":"EACH_ROW"},"autoUpdate":true,"config":{"autoUpdate":true,"promptTemplateDevName":"Customer_Support_Email","promptTemplateType":"flex","modelConfig":{"modelId":"sfdc_ai__DefaultGPT4Omni","modelName":"sfdc_ai__DefaultGPT4Omni"},"promptTemplateInputConfigs":[{"referenceName":"CustomerName","definition":"Customer name","referenceAttribute":{"columnId":"{customer-names-column-id}","columnName":"Customer Names","columnType":"Text"}},{"referenceName":"Issue","definition":"Customer issue","referenceAttribute":{"columnId":"{issues-column-id}","columnName":"Issues","columnType":"Text"}}]}}}' +}) +``` + +**Step 3: Add Evaluation Columns** + +``` +add_column({ + worksheetId: "{worksheetId}", + name: "Coherence", + type: "Evaluation", + config: '{"name":"Coherence","type":"Evaluation","config":{"type":"Evaluation","queryResponseFormat":{"type":"EACH_ROW"},"autoUpdate":true,"config":{"autoUpdate":true,"evaluationType":"COHERENCE","inputColumnReference":{"columnId":"{generated-emails-column-id}","columnName":"Generated Emails","columnType":"PromptTemplate"},"autoEvaluate":true}}}' +}) +``` + +``` +add_column({ + worksheetId: "{worksheetId}", + name: "Completeness", + type: "Evaluation", + config: '{"name":"Completeness","type":"Evaluation","config":{"type":"Evaluation","queryResponseFormat":{"type":"EACH_ROW"},"autoUpdate":true,"config":{"autoUpdate":true,"evaluationType":"COMPLETENESS","inputColumnReference":{"columnId":"{generated-emails-column-id}","columnName":"Generated Emails","columnType":"PromptTemplate"},"autoEvaluate":true}}}' +}) +``` + +--- + +## Pattern 4: Flow/Apex Testing + +**Goal:** Test a Flow with different inputs and extract outputs. + +### Column Setup + +| Order | Column Name | Type | Purpose | +|-------|-------------|------|---------| +| 1 | Subject | Text | Flow input | +| 2 | Description | Text | Flow input | +| 3 | Priority | Text | Flow input | +| 4 | Flow Result | InvocableAction | Execute Flow | +| 5 | Case Id | Reference | Extract output | +| 6 | Status | Reference | Extract output | + +### Step-by-Step Implementation + +**Step 1: Discover the Flow** + +``` +get_invocable_actions() +// Find the Flow you want to test + +describe_invocable_action({ + actionName: "Create_Support_Case", + actionType: "FLOW" +}) +// Returns input/output schema +``` + +**Step 2: Create Input Columns** + +``` +create_workbook_with_worksheet({ + workbookName: "Flow Testing", + worksheetName: "Create Case Tests" +}) +``` + +``` +add_column({ worksheetId: "{worksheetId}", name: "Subject", type: "Text", + config: '{"name":"Subject","type":"Text","config":{"type":"Text","autoUpdate":true,"config":{"autoUpdate":true}}}' }) + +add_column({ worksheetId: "{worksheetId}", name: "Description", type: "Text", + config: '{"name":"Description","type":"Text","config":{"type":"Text","autoUpdate":true,"config":{"autoUpdate":true}}}' }) + +add_column({ worksheetId: "{worksheetId}", name: "Priority", type: "Text", + config: '{"name":"Priority","type":"Text","config":{"type":"Text","autoUpdate":true,"config":{"autoUpdate":true}}}' }) +``` + +**Step 3: Add InvocableAction Column** + +``` +add_column({ + worksheetId: "{worksheetId}", + name: "Flow Result", + type: "InvocableAction", + config: '{"name":"Flow Result","type":"InvocableAction","config":{"type":"InvocableAction","numberOfRows":50,"queryResponseFormat":{"type":"EACH_ROW"},"autoUpdate":true,"config":{"autoUpdate":true,"actionInfo":{"actionType":"FLOW","actionName":"Create_Support_Case","url":"/services/data/v66.0/actions/custom/flow/Create_Support_Case","label":"Create Support Case"},"inputPayload":"{\\\"Subject\\\": \\\"{$1}\\\", \\\"Description\\\": \\\"{$2}\\\", \\\"Priority\\\": \\\"{$3}\\\"}","referenceAttributes":[{"columnId":"{subject-column-id}","columnName":"Subject","columnType":"Text"},{"columnId":"{description-column-id}","columnName":"Description","columnType":"Text"},{"columnId":"{priority-column-id}","columnName":"Priority","columnType":"Text"}]}}}' +}) +``` + +**Step 4: Add Reference Columns to Extract Outputs** + +``` +add_column({ + worksheetId: "{worksheetId}", + name: "Case Id", + type: "Reference", + config: '{"name":"Case Id","type":"Reference","config":{"type":"Reference","queryResponseFormat":{"type":"EACH_ROW"},"autoUpdate":true,"config":{"autoUpdate":true,"referenceColumnId":"{flow-result-column-id}","referenceField":"outputValues.caseId"}}}' +}) +``` + +``` +add_column({ + worksheetId: "{worksheetId}", + name: "Status", + type: "Reference", + config: '{"name":"Status","type":"Reference","config":{"type":"Reference","queryResponseFormat":{"type":"EACH_ROW"},"autoUpdate":true,"config":{"autoUpdate":true,"referenceColumnId":"{flow-result-column-id}","referenceField":"outputValues.status"}}}' +}) +``` + +--- + +## Pattern 5: Multi-Turn Agent Conversation Testing + +**Goal:** Test multi-turn conversations with conversation history. + +### Column Setup + +| Order | Column Name | Type | Purpose | +|-------|-------------|------|---------| +| 1 | Turn 1 Utterance | Text | First user message | +| 2 | Turn 1 Response | Agent | First agent response | +| 3 | Turn 2 Utterance | Text | Follow-up message | +| 4 | Turn 2 Response | Agent | Second response with history | + +### Implementation + +**Step 1: Create First Turn** + +``` +add_column({ worksheetId: "{worksheetId}", name: "Turn 1 Utterance", type: "Text", + config: '{"name":"Turn 1 Utterance","type":"Text","config":{"type":"Text","autoUpdate":true,"config":{"autoUpdate":true}}}' }) +``` + +``` +add_column({ + worksheetId: "{worksheetId}", + name: "Turn 1 Response", + type: "Agent", + config: '{"name":"Turn 1 Response","type":"Agent","config":{"type":"Agent","numberOfRows":50,"queryResponseFormat":{"type":"EACH_ROW"},"autoUpdate":true,"config":{"autoUpdate":true,"agentId":"0XxRM000000xxxxx","agentVersion":"0XyRM000000xxxxx","utterance":"{$1}","utteranceReferences":[{"columnId":"{turn1-utterance-id}","columnName":"Turn 1 Utterance","columnType":"Text"}]}}}' +}) +``` + +**Step 2: Create Second Turn with History** + +``` +add_column({ worksheetId: "{worksheetId}", name: "Turn 2 Utterance", type: "Text", + config: '{"name":"Turn 2 Utterance","type":"Text","config":{"type":"Text","autoUpdate":true,"config":{"autoUpdate":true}}}' }) +``` + +``` +add_column({ + worksheetId: "{worksheetId}", + name: "Turn 2 Response", + type: "Agent", + config: '{"name":"Turn 2 Response","type":"Agent","config":{"type":"Agent","numberOfRows":50,"queryResponseFormat":{"type":"EACH_ROW"},"autoUpdate":true,"config":{"autoUpdate":true,"agentId":"0XxRM000000xxxxx","agentVersion":"0XyRM000000xxxxx","utterance":"{$1}","utteranceReferences":[{"columnId":"{turn2-utterance-id}","columnName":"Turn 2 Utterance","columnType":"Text"}],"conversationHistory":{"columnId":"{turn1-response-id}","columnName":"Turn 1 Response","columnType":"Agent","fieldName":"conversationHistory"}}}}' +}) +``` + +--- + +## Pattern 6: AI Classification with Single Select + +**Goal:** Classify text into categories using AI. + +### Column Setup + +| Order | Column Name | Type | Purpose | +|-------|-------------|------|---------| +| 1 | Customer Feedback | Text | Input text | +| 2 | Sentiment | AI | Classification | +| 3 | Category | AI | Classification | + +### Implementation + +``` +add_column({ worksheetId: "{worksheetId}", name: "Customer Feedback", type: "Text", + config: '{"name":"Customer Feedback","type":"Text","config":{"type":"Text","autoUpdate":true,"config":{"autoUpdate":true}}}' }) +``` + +``` +add_column({ + worksheetId: "{worksheetId}", + name: "Sentiment", + type: "AI", + config: '{"name":"Sentiment","type":"AI","config":{"type":"AI","numberOfRows":50,"queryResponseFormat":{"type":"EACH_ROW"},"autoUpdate":true,"config":{"autoUpdate":true,"mode":"llm","modelConfig":{"modelId":"sfdc_ai__DefaultGPT4Omni","modelName":"sfdc_ai__DefaultGPT4Omni"},"instruction":"Classify the sentiment of this customer feedback: {$1}","referenceAttributes":[{"columnId":"{feedback-column-id}","columnName":"Customer Feedback","columnType":"Text"}],"responseFormat":{"type":"SINGLE_SELECT","options":[{"label":"Positive","value":"positive"},{"label":"Negative","value":"negative"},{"label":"Neutral","value":"neutral"}]}}}}' +}) +``` + +``` +add_column({ + worksheetId: "{worksheetId}", + name: "Category", + type: "AI", + config: '{"name":"Category","type":"AI","config":{"type":"AI","numberOfRows":50,"queryResponseFormat":{"type":"EACH_ROW"},"autoUpdate":true,"config":{"autoUpdate":true,"mode":"llm","modelConfig":{"modelId":"sfdc_ai__DefaultGPT4Omni","modelName":"sfdc_ai__DefaultGPT4Omni"},"instruction":"Categorize this customer feedback: {$1}","referenceAttributes":[{"columnId":"{feedback-column-id}","columnName":"Customer Feedback","columnType":"Text"}],"responseFormat":{"type":"SINGLE_SELECT","options":[{"label":"Product Issue","value":"product"},{"label":"Service Issue","value":"service"},{"label":"Billing Issue","value":"billing"},{"label":"Feature Request","value":"feature"},{"label":"General Inquiry","value":"general"}]}}}}' +}) +``` + +--- + +## Pattern 7: Data Cloud / DMO Enrichment + +**Goal:** Query Data Cloud DMOs and enrich with AI-generated insights. + +### Column Setup + +| Order | Column Name | Type | Purpose | +|-------|-------------|------|---------| +| 1 | Unified Profiles | DataModelObject | Query DMO records | +| 2 | Profile Summary | AI | Generate insights | + +### Step-by-Step Implementation + +**Step 1: Discover Data Cloud Schema** + +``` +get_dataspaces() +// Returns: { dataspaces: [{ name: "default", label: "Default Dataspace" }] } + +get_data_model_objects({ dataspace: "default" }) +// Returns available DMOs in the dataspace + +get_data_model_object_fields({ dataspace: "default", dmoName: "UnifiedIndividual__dlm" }) +// Returns field definitions for the DMO +``` + +**Step 2: Create Workbook and Worksheet** + +``` +create_workbook_with_worksheet({ + workbookName: "Data Cloud Analysis", + worksheetName: "Unified Profiles" +}) +``` + +**Step 3: Add DataModelObject Column** + +``` +add_column({ + worksheetId: "{worksheetId}", + name: "Unified Profiles", + type: "DataModelObject", + config: '{"name":"Unified Profiles","type":"DataModelObject","config":{"type":"DataModelObject","numberOfRows":50,"queryResponseFormat":{"type":"WHOLE_COLUMN","splitByType":"OBJECT_PER_ROW"},"autoUpdate":true,"config":{"autoUpdate":true,"dataModelObjectApiName":"UnifiedIndividual__dlm","dataspaceName":"default","fields":[{"name":"Id__c","type":"string"},{"name":"FirstName__c","type":"string"},{"name":"LastName__c","type":"string"},{"name":"Email__c","type":"string"}]}}}' +}) +``` + +**Step 4: Add AI Enrichment Column** + +``` +add_column({ + worksheetId: "{worksheetId}", + name: "Profile Summary", + type: "AI", + config: '{"name":"Profile Summary","type":"AI","config":{"type":"AI","numberOfRows":50,"queryResponseFormat":{"type":"EACH_ROW"},"autoUpdate":true,"config":{"autoUpdate":true,"mode":"llm","modelConfig":{"modelId":"sfdc_ai__DefaultGPT4Omni","modelName":"sfdc_ai__DefaultGPT4Omni"},"instruction":"Create a brief customer profile summary:\\nName: {$1} {$2}\\nEmail: {$3}","referenceAttributes":[{"columnId":"{profiles-column-id}","columnName":"Unified Profiles","columnType":"DataModelObject","fieldName":"FirstName__c"},{"columnId":"{profiles-column-id}","columnName":"Unified Profiles","columnType":"DataModelObject","fieldName":"LastName__c"},{"columnId":"{profiles-column-id}","columnName":"Unified Profiles","columnType":"DataModelObject","fieldName":"Email__c"}],"responseFormat":{"type":"PLAIN_TEXT","options":[]}}}}' +}) +``` + +--- + +## Pattern 8: List View Import + +**Goal:** Import records from a Salesforce List View and enrich them. + +### Column Setup + +| Order | Column Name | Type | Purpose | +|-------|-------------|------|---------| +| 1 | Records | Object | Import via List View SOQL | +| 2 | Enrichment | AI | Process the records | + +### Step-by-Step Implementation + +**Step 1: Discover List Views** + +``` +get_list_views() +// Returns available list views with IDs + +get_list_view_soql({ + listViewId: "{listViewId}", + sObjectType: "Account" +}) +// Returns: { soql: "SELECT Id, Name, ... FROM Account WHERE ..." } +``` + +**Step 2: Create Object Column with advancedMode** + +Use the SOQL from the list view directly in an Object column's advanced mode: + +``` +add_column({ + worksheetId: "{worksheetId}", + name: "List View Records", + type: "Object", + config: '{"name":"List View Records","type":"Object","config":{"type":"Object","numberOfRows":50,"queryResponseFormat":{"type":"WHOLE_COLUMN","splitByType":"OBJECT_PER_ROW"},"autoUpdate":true,"config":{"autoUpdate":true,"objectApiName":"Account","advancedMode":{"type":"SOQL","inputs":{"queryString":"SELECT Id, Name, Industry, Description FROM Account WHERE Industry = \'Technology\' LIMIT 50"}}}}}' +}) +``` + +--- + +## Pattern 9: Draft Agent Testing + +**Goal:** Test an unpublished (draft) agent before deploying it. + +### Step-by-Step Implementation + +**Step 1: Discover Draft Agents and Their Topics** + +``` +get_agents({ includeDrafts: true }) +// Returns agents including drafts; draft agents have activeVersion but no published version + +get_draft_topics({ + config: '{"id": "0XxRM000000xxxxx", "name": "My Draft Agent"}' +}) +// Returns topic definitions for the draft agent + +get_draft_context_variables({ + config: '{"id": "0XxRM000000xxxxx", "name": "My Draft Agent"}' +}) +// Returns context variables the draft agent expects +``` + +**Step 2: Set Up the Test Using setup_agent_test with isDraft** + +``` +setup_agent_test({ + agentId: "0XxRM000000xxxxx", + agentVersion: "0XyRM000000xxxxx", + utterances: [ + "Test utterance for draft agent", + "Another test case" + ], + workbookName: "Draft Agent Tests", + worksheetName: "Pre-Deploy Validation", + evaluationTypes: ["COHERENCE", "TOPIC_ASSERTION"], + expectedResponses: ["Expected topic 1", "Expected topic 2"], + isDraft: true +}) +``` + +**Step 3: Or Build Manually with isDraft Flag** + +When adding an AgentTest column manually, set `isDraft: true` in the inner config: + +``` +add_column({ + worksheetId: "{worksheetId}", + name: "Draft Agent Output", + type: "AgentTest", + config: '{"name":"Draft Agent Output","type":"AgentTest","config":{"type":"AgentTest","numberOfRows":50,"queryResponseFormat":{"type":"EACH_ROW"},"autoUpdate":true,"config":{"autoUpdate":true,"agentId":"0XxRM000000xxxxx","agentVersion":"0XyRM000000xxxxx","inputUtterance":{"columnId":"{utterance-col-id}","columnName":"Utterances","columnType":"Text"},"contextVariables":[],"isDraft":true,"enableSimulationMode":false}}}' +}) +``` + +**Step 4: Monitor and Compare** + +``` +poll_worksheet_status({ worksheetId: "{worksheetId}" }) + +// Once complete, review results +get_worksheet_summary({ worksheetId: "{worksheetId}" }) +``` + +--- + +## Best Practices + +### Column Ordering + +1. **Input columns first** -- Text, Object columns that provide data +2. **Processing columns next** -- Agent, AI, PromptTemplate, InvocableAction +3. **Extraction columns** -- Reference columns to pull specific fields +4. **Evaluation columns last** -- Depend on processing columns + +### Reference Management + +- Always use the exact column ID returned from the `add_column` response +- Use `fieldName` in ReferenceAttribute to extract specific fields from JSON +- For Object columns, `fieldName` specifies which SObject field to use + +### Error Handling + +- Check column status via `get_worksheet_summary` after processing +- Use `reprocess_column` to retry failed cells +- Use `get_worksheet_data` and inspect cell `statusMessage` for error details + +### State Refresh + +- After any mutation (add_column, paste_data, trigger_row_execution), call `get_worksheet_data` to get updated IDs and statuses +- Use `poll_worksheet_status` for long-running operations instead of manual polling diff --git a/skills/agentforce-grid/references/workflow-patterns.md b/skills/agentforce-grid/references/workflow-patterns.md new file mode 100644 index 0000000..b0f8d7b --- /dev/null +++ b/skills/agentforce-grid/references/workflow-patterns.md @@ -0,0 +1,1161 @@ +# Workflow Patterns and User Interaction Models + +Detailed specifications for how users interact with Agentforce Grid through Claude Code. The interaction model treats grids as manipulable blocks — users describe intent in natural language, and the skill translates that into orchestrated API calls. + +--- + +## 1. Natural Language Grid Building + +### Design Principle + +Users should never think in JSON or API calls. They describe what they want as if talking to a colleague: "Build me a grid that does X." The skill decomposes the request into a plan, confirms it, and executes step-by-step. + +### Conversation Flow: The Three-Phase Pattern + +Every grid-building interaction follows three phases: + +**Phase 1: Understand and Plan** +Parse the user's intent. Identify the column pipeline. Surface any ambiguities. + +**Phase 2: Confirm and Resolve** +Present the plan as a visual table. Ask for missing information (agent IDs, model preferences, filter criteria). Let the user adjust before any API call. + +**Phase 3: Execute and Report** +Create resources sequentially (workbook, worksheet, columns, data, trigger). Report progress. Show the final grid structure. + +### Example A: Agent Testing Grid + +``` +USER: Create a grid that tests my Sales Agent with 50 utterances and + evaluates coherence + topic routing + +SKILL THINKING: + - Need: AgentTest pipeline with evaluations + - Missing: agent ID, agent version, utterance source, expected topics + - Column pipeline: Text (utterances) -> Text (expected topics) -> + AgentTest (run agent) -> Evaluation (COHERENCE) -> Evaluation (TOPIC_ASSERTION) + +SKILL RESPONSE: + I'll build a Sales Agent testing grid with this structure: + + | # | Column | Type | Purpose | + |----|-------------------|------------|----------------------------| + | 1 | Test Utterances | Text | Your 50 test inputs | + | 2 | Expected Topics | Text | Ground truth topic names | + | 3 | Agent Output | AgentTest | Run your Sales Agent | + | 4 | Coherence | Evaluation | Assess logical flow (auto) | + | 5 | Topic Routing | Evaluation | Verify correct topic (auto)| + + Before I create this, I need: + 1. Your agent ID and version — run `sf agent list` or I can call + get_agents to find it. Which agent name should I look for? + 2. Do you already have utterances, or should I create empty rows + for you to fill? + 3. Any context variables your agent requires? + +USER: The agent is called "Sales Assistant". I have utterances in a CSV + at /tmp/sales-tests.csv. No context variables needed. + +SKILL EXECUTES: + 1. get_agents() → find "Sales Assistant" → extract agentId + versionId + 2. create_workbook_with_worksheet({workbookName: "Sales Agent Test Suite", worksheetName: "Sales Assistant Tests"}) + 3. import_csv or create Text columns + paste_data from CSV + 4. add_column → Text column "Expected Topics" + 5. add_column → AgentTest column referencing utterances + 6. add_column → Evaluation COHERENCE on agent output + 7. add_column → Evaluation TOPIC_ASSERTION on agent output, referencing expected topics + 8. trigger_row_execution → process all rows + +SKILL RESPONSE: + Grid created and processing: + + Workbook: "Sales Agent Test Suite" (1W4xx...) + Worksheet: "Sales Assistant Tests" (1W1xx...) + + | Column | ID | Status | + |------------------|-----------|-------------| + | Test Utterances | 1W5xx... | Complete | + | Expected Topics | 1W5xx... | New (empty) | + | Agent Output | 1W5xx... | InProgress | + | Coherence | 1W5xx... | New | + | Topic Routing | 1W5xx... | New | + + 50 rows loaded from CSV. Agent processing started — evaluations + will auto-run as agent responses complete. + + Next steps: + - Fill in "Expected Topics" column with your ground truth + - Use `/grid-status` to check progress + - Once complete, use `/grid-results` to see evaluation scores +``` + +### Example B: Data Enrichment Pipeline + +``` +USER: Build an enrichment pipeline: query tech accounts, generate + summaries, classify sentiment + +SKILL THINKING: + - Need: Object (accounts) -> AI (summaries) -> AI (sentiment classification) + - Missing: specific fields to query, filter details, model preference + - The sentiment column should use SINGLE_SELECT + +SKILL RESPONSE: + Here's the enrichment pipeline I'll build: + + | # | Column | Type | Purpose | + |----|-----------------|--------|-----------------------------------| + | 1 | Tech Accounts | Object | Query Account where Industry = Technology | + | 2 | Summary | AI | Generate company summary from Name, Industry, Description | + | 3 | Sentiment | AI | Classify as Positive/Negative/Neutral | + + A few questions: + 1. Which Account fields matter most? I'd default to Name, Industry, + Description, AnnualRevenue, Website. Add or remove any? + 2. For the summary — how long? (1 sentence, 2-3 sentences, paragraph) + 3. What model? Default is GPT 4 Omni. Options include Claude Sonnet 4.5, + Gemini 2.5 Pro, GPT 5, etc. + 4. How many accounts max? (Default: 50) + +USER: Default fields are fine. 2-3 sentences. Use Claude Sonnet 4.5. 100 accounts. + +SKILL EXECUTES: + 1. create_workbook_with_worksheet({workbookName: "Account Enrichment", worksheetName: "Tech Account Enrichment"}) + 2. add_column → Object column (Account, Industry In ["Technology"], + fields: Id, Name, Industry, Description, AnnualRevenue, Website, + numberOfRows: 100, WHOLE_COLUMN) + 3. Wait for Object column to populate (or proceed — autoUpdate handles it) + 4. add_column → AI column "Summary" (EACH_ROW, instruction referencing + {$1}..{$5}, model: sfdc_ai__DefaultBedrockAnthropicClaude45Sonnet, + PLAIN_TEXT) + 5. add_column → AI column "Sentiment" (EACH_ROW, instruction: + "Classify sentiment of: {$1}", referencing Summary column, + SINGLE_SELECT with Positive/Negative/Neutral) +``` + +### Example C: A/B Model Comparison + +``` +USER: Set up A/B testing: compare GPT-4o vs Claude on customer email generation + +SKILL THINKING: + - Need: Same input → two AI columns with different models → evaluations on both + - This is a side-by-side comparison pattern + - Need a source of customer data or test inputs + +SKILL RESPONSE: + I'll build a side-by-side model comparison grid: + + | # | Column | Type | Purpose | + |----|---------------------|------------|-----------------------------| + | 1 | Customer Name | Text | Input data | + | 2 | Customer Issue | Text | Input data | + | 3 | Email (GPT-4o) | AI | GPT 4 Omni generation | + | 4 | Email (Claude) | AI | Claude Sonnet 4.5 generation| + | 5 | Coherence (GPT-4o) | Evaluation | Quality score for GPT | + | 6 | Coherence (Claude) | Evaluation | Quality score for Claude | + | 7 | Conciseness (GPT-4o)| Evaluation | Brevity score for GPT | + | 8 | Conciseness (Claude)| Evaluation | Brevity score for Claude | + + Both AI columns use the same prompt but different models. + Where should the customer data come from? + a) I'll create empty rows — you paste data in + b) Query from Salesforce Cases/Contacts + c) You have a CSV + +USER: Query from open Cases — use Subject and Description + +SKILL EXECUTES: + 1. create_workbook_with_worksheet({workbookName: "Model Comparison", worksheetName: "GPT vs Claude"}) + 2. add_column → Object column: Case, filters: [Status In ["New","Open"]], + fields: [Subject, Description, ContactId], WHOLE_COLUMN + 3. add_column → AI column "Email (GPT-4o)": model sfdc_ai__DefaultGPT4Omni, + instruction referencing Case.Subject {$1} and Case.Description {$2} + 4. add_column → AI column "Email (Claude)": model + sfdc_ai__DefaultBedrockAnthropicClaude45Sonnet, SAME instruction + referencing same columns + 5. add_column → Evaluation COHERENCE on "Email (GPT-4o)" + 6. add_column → Evaluation COHERENCE on "Email (Claude)" + 7. add_column → Evaluation CONCISENESS on "Email (GPT-4o)" + 8. add_column → Evaluation CONCISENESS on "Email (Claude)" + 9. poll_worksheet_status to monitor execution +``` + +### Translation Rules: Natural Language to API Calls + +| User Says | Skill Understands | API Pattern | +|-----------|-------------------|-------------| +| "test my agent" | AgentTest pipeline | Text + AgentTest + Evaluation columns | +| "query accounts/contacts/leads" | Object column | Object with WHOLE_COLUMN | +| "generate/write/draft" | AI column | AI with mode: "llm", PLAIN_TEXT | +| "classify/categorize" | AI column with SINGLE_SELECT | AI with SINGLE_SELECT response | +| "evaluate/score/check quality" | Evaluation columns | Evaluation with appropriate type | +| "compare X vs Y" | Side-by-side AI columns | Same prompt, different modelConfig | +| "enrich" | Object + AI pipeline | Object (WHOLE_COLUMN) then AI (EACH_ROW) | +| "run this flow/apex" | InvocableAction column | InvocableAction + Reference extraction | +| "extract the topic/action/field" | Reference column | Reference with JSON path | +| "compute/calculate" | Formula column | Formula with expression | + +--- + +## 2. Iterative Grid Refinement + +### Design Principle + +Grids are living documents. Users should be able to modify them conversationally, the same way you'd say "add a column" or "change that filter" to a colleague sitting next to you. + +### Pre-Requisite: Grid Context + +Before any modification, the skill must establish context. If the user says "add a column to my grid," the skill needs the worksheet ID. The conversation flow: + +``` +SKILL: Which grid are you working with? I can: + a) List your workbooks — get_workbooks + b) Use a specific worksheet ID if you have one + c) Use the grid we just created (if in same session) +``` + +Once context is established (worksheet ID known), store it for the session. + +### Workflow 2A: Adding Columns to Existing Grids + +``` +USER: Add a conciseness evaluation to the agent test grid we just built + +SKILL THINKING: + - Have worksheet ID from session context + - Need to: get worksheet data to find the agent output column ID + - Then add Evaluation column with CONCISENESS type + +SKILL EXECUTES: + 1. get_worksheet_data({worksheetId: "..."}) → find "Agent Output" column, get its ID + 2. add_column → Evaluation column: + {evaluationType: "CONCISENESS", inputColumnReference: {agent output col}} + 3. Column auto-processes existing rows + +SKILL RESPONSE: + Added "Conciseness" evaluation column (1W5xx...) targeting "Agent Output". + Auto-evaluating existing 50 rows now. +``` + +### Workflow 2B: Changing Evaluation Criteria + +``` +USER: Change the coherence evaluation to use custom LLM evaluation instead. + I want it to check if the response follows our brand voice guidelines. + +SKILL THINKING: + - Need to update existing evaluation column + - Switch from COHERENCE to CUSTOM_LLM_EVALUATION + - Need a prompt template for brand voice, or create an inline evaluation + - CUSTOM_LLM_EVALUATION requires a referenceColumnReference + +SKILL RESPONSE: + To switch to a custom LLM evaluation for brand voice, I need: + 1. Do you have a prompt template for brand voice evaluation? + (If not, I can help you create evaluation criteria in a Text column) + 2. What does "brand voice" mean specifically? (e.g., professional but + warm, no jargon, always includes next steps) + +USER: No template. Brand voice = professional, empathetic, always suggests + a next action, never uses "unfortunately" + +SKILL EXECUTES: + 1. add_column → Text column "Brand Voice Criteria" with static value + 2. edit_column({columnId: "{coherence-col-id}", config: "..."}) → update to + CUSTOM_LLM_EVALUATION with referenceColumnReference pointing to criteria column + 3. edit_column triggers reprocessing automatically +``` + +### Workflow 2C: Adjusting Object Column Filters + +``` +USER: The accounts grid is only showing Technology companies. Add Finance + and Healthcare too. + +SKILL EXECUTES: + 1. get_worksheet_data({worksheetId: "..."}) → find Object column, read current config + 2. edit_column({columnId: "{object-col-id}", config: "..."}) → update filters: + Industry In ["Technology", "Finance", "Healthcare"] + 3. edit_column triggers reprocessing automatically + +SKILL RESPONSE: + Updated the Accounts filter to include Technology, Finance, and + Healthcare. Reprocessing — this will refresh the data with the + expanded criteria. Downstream AI and Evaluation columns will + automatically reprocess for new rows. +``` + +### Workflow 2D: Re-running with Different Models + +``` +USER: The Claude results look better. Switch the GPT column to use + Claude too and rerun. + +SKILL EXECUTES: + 1. get_worksheet_data({worksheetId: "..."}) → find "Email (GPT-4o)" column + 2. edit_column({columnId: "...", config: "..."}) → update modelConfig to + sfdc_ai__DefaultBedrockAnthropicClaude45Sonnet, rename to "Email (Claude v2)" + 3. edit_column triggers reprocessing automatically + +SKILL RESPONSE: + Switched "Email (GPT-4o)" to Claude Sonnet 4.5 and renamed to + "Email (Claude v2)". Reprocessing all 50 rows. Evaluation columns + will auto-update once generation completes. +``` + +### Workflow 2E: Adding Rows to Existing Grids + +``` +USER: Add 20 more test cases to the agent grid + +SKILL EXECUTES: + 1. add_rows({worksheetId: "...", numberOfRows: 20}) + 2. Return row IDs + +SKILL RESPONSE: + Added 20 empty rows (total now 70). The new rows have empty + utterances — fill them in and the AgentTest + Evaluation columns + will auto-process. + + To paste data: provide your utterances and I'll use the paste endpoint. + Or update cells individually. +``` + +--- + +## 3. Data Import/Export Workflows + +### Workflow 3A: CSV Import + +**Path 1: Direct CSV Import (API endpoint)** + +``` +USER: Import my test cases from /tmp/agent-tests.csv + +SKILL THINKING: + - The MCP server has import_csv tool + - Needs documentId (the CSV must be uploaded to Salesforce as a + ContentDocument first) and includeHeaders flag + - Alternative: parse CSV locally, use paste_data tool + +SKILL RESPONSE: + Two options for importing your CSV: + + a) **API CSV Import** — Upload the CSV to Salesforce first (as a + ContentDocument), then I call the import-csv endpoint. This + creates Text columns automatically from CSV headers. + + b) **Paste Import** — I read the CSV locally, create Text columns + matching headers, add rows, and paste data via the matrix endpoint. + More control over column naming and no Salesforce upload needed. + + Which approach? (For Claude Code, option b is usually smoother) + +USER: Option b + +SKILL EXECUTES: + 1. Read CSV file locally → parse headers and rows + 2. add_column → Text columns matching each CSV header + 3. add_rows({worksheetId: "...", numberOfRows: 50}) + 4. get_worksheet_data({worksheetId: "..."}) → get column IDs and row IDs + 5. paste_data({worksheetId: "...", startColumnId: "first-col", + startRowId: "first-row", matrix: "[[{...}], ...]"}) + +SKILL RESPONSE: + Imported 50 rows x 3 columns from agent-tests.csv: + - "Utterance" (1W5xx...) — 50 values loaded + - "Expected Response" (1W5xx...) — 50 values loaded + - "Expected Topic" (1W5xx...) — 50 values loaded + + Ready to add processing columns. Want me to set up the agent test pipeline? +``` + +**Path 2: Paste from Description** + +``` +USER: Here are my test utterances: + - "How do I reset my password?" + - "What's my account balance?" + - "I want to cancel my subscription" + - "Transfer me to a human" + - "What are your business hours?" + +SKILL EXECUTES: + 1. Parse the 5 utterances from the message + 2. Ensure Text column exists (or add_column → create one) + 3. add_rows({worksheetId: "...", numberOfRows: 5}) + 4. get_worksheet_data({worksheetId: "..."}) → get row IDs + 5. paste_data → single-column matrix + +SKILL RESPONSE: + Loaded 5 test utterances into the "Test Utterances" column. + Rows 1-5 populated. Processing columns will auto-run. +``` + +### Workflow 3B: Export Results + +``` +USER: Export the evaluation results to CSV + +SKILL THINKING: + - get_worksheet_data → get all cell data + - Format as CSV locally + - No native export endpoint — must reconstruct from cell data + +SKILL EXECUTES: + 1. get_worksheet_data → full worksheet data + 2. Parse columnData: map column IDs to names, row IDs to row numbers + 3. Build CSV: header row from column names, data rows from displayContent + 4. Write to local file + +SKILL RESPONSE: + Exported to /tmp/sales-agent-results.csv + + Summary: + - 50 rows x 5 columns + - Coherence scores: avg 4.2/5, min 2.8, max 5.0 + - Topic routing accuracy: 92% (46/50 correct) + - 4 failed rows (agent timeout) — see rows 12, 23, 37, 41 +``` + +### Workflow 3C: Generate Report from Evaluation Data + +``` +USER: Give me a summary report of the evaluation results + +SKILL EXECUTES: + 1. get_worksheet_data → all cell data + 2. Parse evaluation columns → extract scores/pass-fail + 3. Compute aggregates locally + +SKILL RESPONSE: + ## Sales Agent Evaluation Report + + **Grid:** Sales Agent Test Suite > Sales Assistant Tests + **Date:** 2026-03-06 + **Rows:** 50 test cases + + ### Evaluation Summary + + | Metric | Result | Details | + |---------------------|-------------|----------------------| + | Coherence (avg) | 4.2 / 5.0 | Range: 2.8 - 5.0 | + | Topic Routing | 92% pass | 46/50 correct | + | Response Match | 78% pass | 39/50 matched | + | Latency | 100% pass | All under threshold | + + ### Worst Performers (Bottom 5 by Coherence) + | Row | Utterance | Score | Topic | + |-----|----------------------------------|-------|---------| + | 12 | "complex multi-part question..." | 2.8 | Wrong | + | 23 | "edge case scenario..." | 3.0 | Correct | + | 37 | "ambiguous request..." | 3.1 | Wrong | + | 8 | "slang/informal input..." | 3.2 | Correct | + | 41 | "very long utterance..." | 3.3 | Wrong | + + ### Recommendations + - 3 of bottom 5 also had wrong topic routing — investigate topic + classification for complex/ambiguous inputs + - Row 12 (complex multi-part) suggests agent struggles with + compound questions — consider adding topic for multi-intent +``` + +--- + +## 4. Monitoring and Observability Workflows + +### Workflow 4A: Status Check — "How's my grid doing?" + +``` +USER: How's my grid doing? + +SKILL EXECUTES: + 1. get_worksheet_data → check cell statuses + 2. Count by status per column: Complete, InProgress, Failed, New + +SKILL RESPONSE: + ## Grid Status: Sales Assistant Tests + + | Column | Complete | InProgress | Failed | New | + |------------------|----------|------------|--------|------| + | Test Utterances | 50 | 0 | 0 | 0 | + | Expected Topics | 50 | 0 | 0 | 0 | + | Agent Output | 43 | 5 | 2 | 0 | + | Coherence | 43 | 0 | 0 | 7 | + | Topic Routing | 43 | 0 | 0 | 7 | + + Overall: 86% complete. 5 agent calls still running, 2 failed. + Evaluations are pending on the 7 incomplete agent rows. + + Estimated completion: ~2 minutes (based on current processing rate). +``` + +### Workflow 4B: Debugging Failures — "Why did row 5 fail?" + +``` +USER: Why did row 12 fail? + +SKILL EXECUTES: + 1. get_worksheet_data → find row 12 cells + 2. Check each column's cell for row 12 + 3. Look at status and statusMessage for Failed cells + 4. Look at fullContent for error details + +SKILL RESPONSE: + Row 12 failure analysis: + + | Column | Status | Detail | + |-----------------|----------|-------------------------------------| + | Test Utterances | Complete | "I need to reset my password and | + | | | also check my balance and also..." | + | Agent Output | Failed | Error: "Agent response timeout. | + | | | Execution exceeded 120s limit." | + | Coherence | Skipped | Skipped (no agent output) | + | Topic Routing | Skipped | Skipped (no agent output) | + + **Root cause:** The utterance is very long and multi-part (148 words). + The agent likely entered a complex reasoning loop and timed out. + + **Suggested fixes:** + 1. Simplify the utterance — split into separate test cases + 2. Reprocess just this row: reprocess_column({columnId: "{agent-col-id}"}) + 3. If this is a real user scenario, the agent itself may need + optimization for compound queries +``` + +### Workflow 4C: Analysis — "Show me the worst-performing test cases" + +``` +USER: Show me the worst-performing test cases + +SKILL EXECUTES: + 1. get_worksheet_data → all evaluation cell data + 2. Parse evaluation scores from displayContent + 3. Sort by score ascending, take bottom N + 4. Cross-reference with input utterances + +SKILL RESPONSE: + ## Bottom 10 Test Cases by Evaluation Score + + | Row | Utterance (truncated) | Coherence | Topic | Match | + |-----|--------------------------------|-----------|--------|-------| + | 12 | "reset password and check b..."| Failed | -- | -- | + | 37 | "what if my payment was rej..."| 2.1 | FAIL | FAIL | + | 23 | "can you do a thing for me" | 2.8 | FAIL | FAIL | + | 41 | "URGENT: need help NOW!!!" | 3.0 | PASS | FAIL | + | 8 | "yo whats my bal" | 3.2 | PASS | FAIL | + | 15 | "I was told by your agent t..."| 3.3 | FAIL | PASS | + | 29 | "" | 3.4 | FAIL | -- | + | 33 | "repeat: transfer me" | 3.5 | PASS | FAIL | + | 44 | "hola necesito ayuda" | 3.5 | FAIL | FAIL | + | 19 | "what is the meaning of life" | 3.6 | FAIL | PASS | + + **Patterns detected:** + - Informal/slang inputs score lower (rows 8, 41) + - Ambiguous inputs cause topic routing failures (rows 23, 19) + - Non-English input not handled (row 44) + - Empty input causes failure (row 29) + - Complex multi-part questions timeout (row 12) +``` + +### Workflow 4D: Polling Pattern for Long-Running Grids + +For grids with many rows or expensive operations (agent calls, LLM evaluations), the skill should implement polling: + +``` +SKILL (internal polling logic): + + Poll Strategy: + 1. After triggering execution, wait 10 seconds + 2. get_worksheet_data → count statuses + 3. If any InProgress: report progress, wait 15 seconds, poll again + 4. If all Complete/Failed: report final results + 5. Max poll attempts: 20 (5 minutes total) + 6. If still running after max: report current state, suggest user + check back with /grid-status + + Progress Reporting (during long operations): + "Processing: 23/50 complete (46%). 27 still running..." + "Processing: 41/50 complete (82%). 7 still running, 2 failed..." + "Complete: 48/50 succeeded, 2 failed. Generating report..." +``` + +**Important:** The API does not provide webhooks or push notifications. All monitoring is poll-based. Use `poll_worksheet_status` for automated polling with built-in intervals, or `get_worksheet_summary` for one-time status checks. Manual `get_worksheet_data` polling is also supported but requires managing frequency to avoid rate limits. + +### Workflow 4E: Stale Detection and Auto-Reprocess + +``` +USER: I updated some utterances. Why aren't the evaluations updating? + +SKILL EXECUTES: + 1. get_worksheet_data → check for Stale status cells + 2. Identify which columns need reprocessing + +SKILL RESPONSE: + Found 5 rows with stale cells — the agent output and evaluations + haven't re-run since you changed the utterances. + + | Rows | Stale Columns | + |-------|----------------------------------------| + | 3,7,12,28,45 | Agent Output, Coherence, Topic Routing | + + Want me to reprocess? This will re-run the agent and evaluations + for these 5 rows. + +USER: Yes + +SKILL EXECUTES: + 1. trigger_row_execution + → {"rowIds": ["row-3", "row-7", "row-12", "row-28", "row-45"]} +``` + +--- + +## 5. CI/CD Integration Patterns + +### Design Principle + +Grids become automated quality gates. An agent's evaluation scores must meet thresholds before deployment. The skill helps users set up grids that can be triggered programmatically and produce machine-readable results. + +### Workflow 5A: Evaluation as a Deployment Gate + +``` +USER: I want to run grid evaluations as part of our deployment pipeline. + How should I set that up? + +SKILL RESPONSE: + Here's the recommended CI/CD integration pattern: + + ## Architecture + + ``` + CI Pipeline + | + v + 1. sf deploy (agent metadata) + | + v + 2. Claude Code: /grid-run {worksheet-id} + | + v + 3. Poll for completion + | + v + 4. Claude Code: /grid-results {worksheet-id} --format json + | + v + 5. Assert: coherence_avg >= 4.0 + Assert: topic_accuracy >= 90% + Assert: failed_rows == 0 + | + v + 6. Pass → continue deploy / Fail → block + report + ``` + + ## Script Template (bash) + + ```bash + #!/bin/bash + # Run after agent deployment + + WORKSHEET_ID="1W1xx0000004xxxx" + BASE_URL="/services/data/v66.0/public/grid" + + # Step 1: Reprocess the agent column to test with latest agent version + AGENT_COL_ID="1W5xx0000004xxxx" + sf api request "$BASE_URL/columns/$AGENT_COL_ID/reprocess" --method POST + + # Step 2: Poll until complete + while true; do + STATUS=$(sf api request "$BASE_URL/worksheets/$WORKSHEET_ID/data" \ + | jq '[.columnData[][].status] | group_by(.) | map({(.[0]): length}) | add') + PENDING=$(echo $STATUS | jq '.InProgress // 0') + if [ "$PENDING" -eq 0 ]; then break; fi + sleep 15 + done + + # Step 3: Extract evaluation scores + # (parse from worksheet data response) + + # Step 4: Assert thresholds + ``` + + Want me to create a reusable test grid for your agent that you can + trigger from CI? +``` + +### Workflow 5B: Version Comparison + +``` +USER: I just deployed a new version of my Sales Agent. Compare it + against the previous version's results. + +SKILL THINKING: + - Strategy: Create a new worksheet in same workbook + - Copy the same utterances from the existing test sheet + - Run against the new agent version + - Compare scores side-by-side + +SKILL EXECUTES: + 1. get_workbooks → find the agent test workbook + 2. get_worksheet_data({worksheetId: "{old-id}"}) → extract utterances + expected values + 3. create_worksheet → new worksheet "Sales Assistant v2.1 Tests" + 4. Create same column structure but with new agent version + 5. Paste same utterances + 6. Trigger execution + 7. When complete: compare results + +SKILL RESPONSE: + ## Version Comparison: Sales Assistant v2.0 vs v2.1 + + | Metric | v2.0 | v2.1 | Delta | + |---------------------|-------------|-------------|----------| + | Coherence (avg) | 4.2 / 5.0 | 4.5 / 5.0 | +0.3 | + | Topic Routing | 92% (46/50) | 96% (48/50) | +4% | + | Response Match | 78% (39/50) | 82% (41/50) | +4% | + | Latency Pass Rate | 100% | 98% (49/50) | -2% | + | Failures | 2 | 1 | -1 | + + **Regression detected:** Latency — row 37 now exceeds threshold + (was 1.8s, now 2.3s). Investigate if new topic logic adds latency. + + **Improvements:** Topic routing fixed for rows 23, 37 (previously + wrong, now correct). Coherence improved across the board. +``` + +### Workflow 5C: Regression Detection + +The skill should flag any metric that degrades between versions: + +``` +SKILL (automated regression analysis): + + Regression Rules: + 1. Any evaluation average that drops > 0.2 points: WARNING + 2. Any evaluation average that drops > 0.5 points: FAILURE + 3. Any previously-passing row that now fails: FLAG + 4. New failures in previously-stable test cases: CRITICAL + 5. Latency increase > 20%: WARNING + + Output format (machine-readable for CI): + { + "status": "WARNING", + "regressions": [ + { + "metric": "latency_pass_rate", + "previous": 1.0, + "current": 0.98, + "delta": -0.02, + "severity": "WARNING", + "affected_rows": [37] + } + ], + "improvements": [ + { + "metric": "coherence_avg", + "previous": 4.2, + "current": 4.5, + "delta": 0.3 + } + ] + } +``` + +### Workflow 5D: Connection to Agent Refinery Loop + +``` + The Grid-Refinery Feedback Loop: + + 1. GRID (Evaluate) + - Run test suite against current agent + - Identify failure patterns + + 2. REFINERY (Improve) + - Use failure analysis to refine agent topics + - Update instructions, actions, guardrails + - Deploy new agent version + + 3. GRID (Re-Evaluate) + - Run same test suite against updated agent + - Compare: did failures resolve? + - Did anything regress? + + 4. Repeat until quality gates pass + + The skill's role: automate steps 1 and 3, surface insights + that inform step 2. +``` + +--- + +## 6. Multi-Grid Orchestration + +### Workflow 6A: Related Worksheets in One Workbook + +``` +USER: I need a complete test suite: one sheet for happy path tests, + one for edge cases, one for adversarial inputs. + +SKILL EXECUTES: + 1. create_workbook({name: "Sales Agent Complete Test Suite"}) + 2. create_worksheet → "Happy Path Tests" (workbookId: ...) + 3. create_worksheet → "Edge Case Tests" (workbookId: ...) + 4. create_worksheet → "Adversarial Tests" (workbookId: ...) + 5. For each: create identical column structure (Text utterances → + AgentTest → Evaluations) + 6. Populate each with appropriate test cases + +SKILL RESPONSE: + Created test suite with 3 worksheets: + + **Workbook:** Sales Agent Complete Test Suite (1W4xx...) + + | Worksheet | ID | Rows | Purpose | + |----------------------|----------|------|----------------------| + | Happy Path Tests | 1W1xx... | 30 | Standard scenarios | + | Edge Case Tests | 1W1xx... | 20 | Boundary conditions | + | Adversarial Tests | 1W1xx... | 15 | Malicious/weird input| + + All 65 test cases queued for processing. Use `/grid-status` with + any worksheet ID to check progress, or I can report on all three. +``` + +### Workflow 6B: Passing Data Between Grids + +The API does not natively support cross-worksheet references. The skill bridges this gap: + +``` +USER: Use the accounts from my enrichment grid as context for + agent testing in a different worksheet. + +SKILL THINKING: + - Can't reference across worksheets in the API + - Strategy: extract data from worksheet A, paste into worksheet B + - This is a manual data pipeline the skill orchestrates + +SKILL EXECUTES: + 1. get_worksheet_data({worksheetId: "{enrichment-id}"}) → extract Account Names, + Summaries from the enrichment grid + 2. In agent test worksheet: create Text column "Account Context" + 3. Paste the enrichment data into the agent test worksheet + 4. Configure Agent column to use "Account Context" as a + context variable + +SKILL RESPONSE: + Copied 50 account summaries from "Account Enrichment" into + "Agent Test" worksheet as context. The Agent column now receives + each account summary as the AccountContext variable. +``` + +### Workflow 6C: Aggregate Results Across Worksheets + +``` +USER: Give me a combined report across all three test worksheets + +SKILL EXECUTES: + 1. get_workbooks/{id} → list all worksheets + 2. For each worksheet: get_worksheet_data + 3. Parse evaluation results from each + 4. Aggregate + +SKILL RESPONSE: + ## Combined Test Suite Report + + | Worksheet | Rows | Coherence | Topic | Match | Failures | + |-------------------|------|-----------|--------|--------|----------| + | Happy Path | 30 | 4.6 avg | 97% | 90% | 0 | + | Edge Cases | 20 | 3.8 avg | 80% | 65% | 2 | + | Adversarial | 15 | 3.2 avg | 60% | 40% | 5 | + | **Overall** | **65** | **4.0 avg** | **83%** | **70%** | **7** | + + The agent handles happy path well but struggles with adversarial + inputs. Key weakness: topic routing under adversarial conditions + (60% accuracy). +``` + +--- + +## 7. Slash Commands + +### Command Reference + +| Command | Description | Arguments | +|---------|-------------|-----------| +| `/grid-new` | Create new grid from description | `` | +| `/grid-status` | Show current grid state | `[worksheet-id]` (optional if in session) | +| `/grid-run` | Execute/reprocess grid | `[worksheet-id] [--rows ] [--column ]` | +| `/grid-results` | Show evaluation results | `[worksheet-id] [--format table\|json\|csv]` | +| `/grid-add` | Add column to existing grid | `` | +| `/grid-debug` | Investigate failures | `[row-number] [--column ]` | +| `/grid-compare` | Compare two worksheets | ` ` | +| `/grid-export` | Export grid data | `[worksheet-id] [--format csv\|json] [--path ]` | +| `/grid-list` | List workbooks and worksheets | | +| `/grid-models` | List available LLM models | | + +### `/grid-new` — Create New Grid from Description + +``` +Usage: /grid-new + +Examples: + /grid-new Test my Support Agent with 50 utterances, evaluate coherence and topic routing + /grid-new Enrich tech accounts with AI summaries and sentiment classification + /grid-new Compare GPT-4o vs Claude on email drafting for open Cases + /grid-new Run my Create_Case flow with 20 test inputs and extract CaseId + +Behavior: + 1. Parse the description into a column pipeline plan + 2. Present the plan as a table for confirmation + 3. Ask for any missing information (agent IDs, model preferences, etc.) + 4. On confirmation: create workbook, worksheet, columns, data + 5. Trigger processing + 6. Report grid structure with IDs +``` + +### `/grid-status` — Show Current Grid State + +``` +Usage: /grid-status [worksheet-id] + +If worksheet-id omitted: uses the worksheet from current session context, +or lists workbooks and asks user to pick. + +Output: + - Per-column status breakdown (Complete/InProgress/Failed/New counts) + - Overall completion percentage + - Estimated time remaining (if InProgress rows exist) + - List of failed rows (if any) + +Examples: + /grid-status + /grid-status 1W1xx0000004xxxx +``` + +### `/grid-run` — Execute/Reprocess + +``` +Usage: /grid-run [options] + +Options: + --all Reprocess entire grid (all columns) + --column Reprocess specific column + --rows Process specific rows only + --failed Reprocess only failed rows + --stale Reprocess only stale rows + +Examples: + /grid-run # Trigger all pending rows + /grid-run --failed # Retry failures + /grid-run --column "Agent Output" # Reprocess agent column + /grid-run --rows row1,row2,row3 # Process specific rows + +API calls: + --all / --column → reprocess_column + --rows → trigger_row_execution {rowIds: [...]} + --failed → GET data, find Failed rows, then trigger-row-execution + --stale → GET data, find Stale rows, then trigger-row-execution +``` + +### `/grid-results` — Show Evaluation Results + +``` +Usage: /grid-results [worksheet-id] [options] + +Options: + --format table|json|csv Output format (default: table) + --sort asc|desc Sort by evaluation column + --top Show top N results + --bottom Show bottom N results + --failed-only Show only failed evaluations + --summary Show aggregate summary only + +Examples: + /grid-results # Full results table + /grid-results --summary # Just averages and pass rates + /grid-results --bottom 10 # Worst 10 performers + /grid-results --format json # Machine-readable output + /grid-results --failed-only --sort coherence asc + +API calls: + 1. get_worksheet_data + 2. Parse evaluation columns + 3. Format per options +``` + +### `/grid-add` — Add Column to Existing Grid + +``` +Usage: /grid-add + +Examples: + /grid-add evaluation for conciseness + /grid-add AI column that classifies urgency as High/Medium/Low + /grid-add Reference column extracting topicName from Agent Output + /grid-add Formula: CONCATENATE first name and last name + +Behavior: + 1. Identify target worksheet (from session or ask) + 2. Parse column description → determine type and config + 3. GET worksheet to find existing columns (for references) + 4. Confirm plan with user + 5. POST column + 6. Report result +``` + +### `/grid-debug` — Investigate Failures + +``` +Usage: /grid-debug [row-number] [--column ] + +Examples: + /grid-debug 12 # Full row 12 analysis + /grid-debug --column "Agent Output" # All failures in that column + /grid-debug # Summary of all failures + +Behavior: + 1. get_worksheet_data + 2. Find relevant cells by row number or column + 3. Extract status, statusMessage, fullContent for failed cells + 4. Present diagnosis with suggested fixes +``` + +### `/grid-compare` — Compare Two Worksheets + +``` +Usage: /grid-compare + +Examples: + /grid-compare 1W1xx001 1W1xx002 # Compare v1 vs v2 results + +Behavior: + 1. GET data from both worksheets + 2. Match rows by utterance text (or row order) + 3. Compare evaluation scores column by column + 4. Flag regressions and improvements + 5. Output comparison table + delta analysis +``` + +### `/grid-export` — Export Grid Data + +``` +Usage: /grid-export [options] + +Options: + --format csv|json Output format (default: csv) + --path Write to file (default: /tmp/grid-export-{timestamp}.csv) + --columns Export only specific columns + +Examples: + /grid-export --format csv --path /tmp/results.csv + /grid-export --format json + /grid-export --columns "Utterance,Agent Output,Coherence" +``` + +### `/grid-list` — List Workbooks and Worksheets + +``` +Usage: /grid-list + +Behavior: + 1. get_workbooks + 2. For each workbook: show worksheets with basic info + +Output: + Workbooks: + 1. Sales Agent Test Suite (1W4xx...) + - Sales Assistant Tests (1W1xx...) — 50 rows, 5 columns + - Sales Assistant v2.1 Tests (1W1xx...) — 50 rows, 5 columns + 2. Account Enrichment (1W4xx...) + - Tech Account Enrichment (1W1xx...) — 100 rows, 3 columns +``` + +### `/grid-models` — List Available Models + +``` +Usage: /grid-models + +Behavior: + 1. get_llm_models + 2. Format as table with name, label, max tokens + +Output: + | Model ID | Label | Max Tokens | + |---------------------------------------------|--------------------|------------| + | sfdc_ai__DefaultGPT4Omni | GPT 4 Omni | 16384 | + | sfdc_ai__DefaultGPT5 | GPT 5 | 128000 | + | sfdc_ai__DefaultBedrockAnthropicClaude45Sonnet | Claude Sonnet 4.5 | 8192 | + | sfdc_ai__DefaultVertexAIGemini25Flash001 | Gemini 2.5 Flash | 65536 | + | ... | | | +``` + +--- + +## 8. Conversation State Management + +### Session Context + +The skill should maintain context within a session to avoid repetitive questions: + +``` +Session State (maintained across turns): + - active_workbook_id: "1W4xx..." (set after create or select) + - active_worksheet_id: "1W1xx..." (set after create or select) + - column_map: {name -> id} (refreshed after column operations) + - row_ids: [...] (refreshed after row operations) + - last_agent_id: "0Xxxx..." (reused for same-agent operations) + - last_model: "sfdc_ai__DefaultGPT4Omni" (reused for defaults) +``` + +### Disambiguation Pattern + +When context is ambiguous, ask — don't guess: + +``` +USER: Add an evaluation column + +SKILL: Add an evaluation to which grid? + 1. Sales Assistant Tests (1W1xx...) — has Agent Output column + 2. Tech Account Enrichment (1W1xx...) — has AI Summary column + + And which evaluation type? + - Quality: COHERENCE, CONCISENESS, FACTUALITY, COMPLETENESS, + INSTRUCTION_FOLLOWING + - Comparison: RESPONSE_MATCH, TOPIC_ASSERTION, ACTION_ASSERTION, + BOT_RESPONSE_RATING + - Performance: LATENCY_ASSERTION + - Custom: EXPRESSION_EVAL, CUSTOM_LLM_EVALUATION +``` + +### Error Recovery Pattern + +When API calls fail, the skill should diagnose and suggest fixes: + +``` +API Error: 400 BAD_REQUEST + "config.config.mode is required for AI column type" + +SKILL RESPONSE: + The AI column creation failed — I missed a required field in the + config. Fixing and retrying... + + [Retries with mode: "llm" added] + + Fixed. Column "Summary" created successfully. +``` + +``` +API Error: 404 NOT_FOUND + "Agent with ID 0Xxxx not found" + +SKILL RESPONSE: + That agent ID doesn't exist. Let me look up your available agents. + [get_agents] + + Found these agents: + 1. Sales Assistant (0Xxxx001) — Active, v3 + 2. Support Bot (0Xxxx002) — Active, v1 + 3. Onboarding Agent (0Xxxx003) — Draft + + Which one should I use? +```