Skill Eval

Overview

This skill owns the repository's skill-eval operation through skills/skill-eval/scripts/eval_runner.py: validating, running, grading, aggregating, and reporting suites under evals/<skill-name>/, and verifying a result before it is reported. The runner keeps executor and grader separate and records artifacts and metrics. Never hand-run prompts, hand-record results, let one agent execute and grade the same answer, or estimate metrics. Deciding which skill or eval contract to change from a result belongs to the eval-quality workflow, not to this skill.

Eval Run Authorization

Do not launch eval_runner.py run, provider subprocesses, or a new iteration unless the current user explicitly asks to run evals, run a benchmark, or execute the runner. Editing a skill, inspecting results, validating a suite, or proving quality does not implicitly authorize a fresh run.