skill-evaluator
Installation
SKILL.md
Skill Evaluator
Test whether a skill is correct, discoverable, and valuable — not just whether its unit tests pass. The harness is agent-agnostic: it drives whichever coding-agent CLI the user uses, because Agent Skills are portable across all of them.
When to use which layer
Three layers, increasing cost and fidelity (full rationale in
references/methodology.md):
| Layer | Question | Script | Needs a CLI? |
|---|---|---|---|
| 1. Deterministic | Do the scripts emit the documented numbers? | run_script_checks.py |
No |
| 2. Trigger | Does the description activate on the right prompts? | run_trigger_eval.py |
Yes |
| 3. Quality | Does following the SKILL.md beat no skill? | run_quality_eval.py → grade → aggregate_benchmark.py |
Yes |
Always run Layer 1 (it's free). Add Layers 2–3 when you can run a coding-agent CLI.