Purpose

This skill produces the Generate artifact of the /eval-guide lifecycle: importable test cases for Copilot Studio's Evaluation tab plus a .docx test-case report carrying the full manifest for human review and downstream Run/Interpret stages. It is the standalone form of /eval-guide Generate.

In the canonical Practical Guidance on Agent Evaluation: 10-step playbook, this skill delivers Step 2 — Build the Capability Eval Sets and Step 3 — Build the Trust & Safety Eval Sets, and it designs the Step 8 — Regression Suite partition for those sets. Keep the operational stage name Generate as UX scaffolding; use the playbook terms for methodology.

Primary mode — the conversation or attachments contain the populated /eval-suite-planner workbook (eval-suite-<agent-name>-<date>.xlsx). Use 2 . Eval Suite Registry as the source of truth for eval sets, and 1 . Planning for risk tier, owners, gates, lifecycle stage, and source dependencies. Generate one set of cases per capability row and one set per trust & safety row. If only a narrative plan is available, use it as a fallback source.

Fallback mode — no plan in conversation. Accept a plain-English agent description and generate test cases from scratch (6–8 cases minimum), using the same data model and including at least one adversarial / trust & safety scenario.

Maturity callout — Pillar 2 (Build your eval sets): Generate advances Pillar 2 from L100 Initial ("no established eval set") to L300 Systematic ("versioned eval set with coverage purposefully targeted"). The CSV files plus companion manifest are the Pillar 2 artifact. The Step 8 partition also seeds Pillars 3 and 5 for later operation.

Instructions

When invoked as /eval-generator (with or without input):

Step 0 — Detect input mode

Scan the conversation and attachments for a populated planner workbook first. If present, read: