eval-skill
Installation
SKILL.md
Eval a Skill
Measure routing and quality separately. A skill that never loads and a skill that loads but adds nothing fail for different reasons.
- Routing: Does the agent load it on prompts it owns and stay quiet on near misses? The name, description, catalog and policy context, competing skills, agent, and model can all affect selection. The body contributes only after the skill loads.
- Quality: With the skill explicitly available, does the result beat the previous skill or no skill? Use observable postconditions and a no-skill arm.
Start with the model that will perform the real task. Run other supported target models separately. A cheaper model is useful for iteration or as an additional stress case, but its result does not predict a stronger model's result.