agent-eval-loop
Installation
SKILL.md
Agent Eval Loop
This is the agent-work equivalent of TDD's red-green-refactor loop: no prompt/RAG/graph change ships without a before/after comparison against a golden set.
If no golden set exists yet
- Ask for 10-20 real (or realistic) queries the agent should handle well, plus what a good answer or correct tool call looks like for each.
- Store them under
evals/<agent-name>/golden-set.jsonl(one JSON object per line:{"input": ..., "expected": ...}). - This is a starting point, not a finished suite — say so to the user, and suggest growing it from real failures over time.