agent-evaluation
Installation
SKILL.md
Agent Evaluation
Agent evaluation measures whether an AI agent produces output that meets a defined quality bar. Without evaluation, you're guessing whether the agent works. With evaluation, you know: the agent passes 94% of accuracy checks, violates zero banned-phrase rules, and produces output rated 4.2/5 by human reviewers. Evaluation turns "it seems fine" into a measurable quality system.
The principle: define what "good" means before building the agent. Write the eval before the prompt. If you can't describe what a good output looks like in measurable terms, you can't evaluate whether the agent produces it.
The Evaluation Framework
4 dimensions of agent quality
| Dimension | What it measures | How to measure | Non-negotiable? |
|---|---|---|---|
| Accuracy | Are the facts correct? No hallucinations, no fabricated data | Cross-check output against input data. Human spot-check | Yes. 95%+ required |
| Compliance | Does output follow all rules? Word limits, banned phrases, format, style | Automated rule checker | Yes. 100% required |
| Completeness | Are all required fields populated? No missing data | Automated schema check | Yes. 90%+ required |
| Quality | Is the output genuinely good? Would a human use it without editing? | Human rating (1-5 scale) | Target ≥ 4.0 average |