Eval Coach

An Agent Skill for designing comprehensive AI evaluation strategies using Evaluation-Driven Development (EDD).

Overview

Eval Coach guides you through a structured 5-step framework for evaluating LLM applications:

1. Define Success - Map business goals to measurable metrics
2. Design Dataset - Create diverse test cases (happy path, edge cases, adversarial)
3. Select Methods - Choose Automated, LLM-as-Judge, or Human evaluation
4. Plan Automation - Integrate evals into CI/CD
5. Monitor Production - Track drift and collect feedback

When to Use This Skill