Creating Evals for a Skill

Generates an arc-skill-eval test suite (evals/evals.json) for an existing skill so authors can prove — and show users — that the skill works. Runs in five phases. Stop and confirm with the user at each phase boundary.

Format reference: docs/evals-json-pivot.md ("Authoring format", "Assertion grading contract") and src/evals/types.ts in this repo.

Phase 1 — Locate and summarize the target skill

1. Confirm the target skill directory. Default to the nearest ancestor containing SKILL.md. Ask if ambiguous.
2. Read the target SKILL.md. Extract:
   • name (from YAML frontmatter)
   • description — used to derive trigger-phrase candidates for routing-style cases
   • numbered phases / steps — each step is a candidate execution case
   • tools the skill expects to call (Read / Bash / Glob / Write / Edit / Grep)
   • files the skill creates, edits, or reads
3. Write a one-paragraph summary back to the user. Ask: "Did I capture the skill's behavior correctly?" Iterate until they confirm before moving on.