agent-eval-loop

Installation
SKILL.md

Agent Eval Loop

This is the agent-work equivalent of TDD's red-green-refactor loop: no prompt/RAG/graph change ships without a before/after comparison against a golden set.

If no golden set exists yet

  1. Ask for 10-20 real (or realistic) queries the agent should handle well, plus what a good answer or correct tool call looks like for each.
  2. Store them under evals/<agent-name>/golden-set.jsonl (one JSON object per line: {"input": ..., "expected": ...}).
  3. This is a starting point, not a finished suite — say so to the user, and suggest growing it from real failures over time.

Running the loop

Installs
1
First Seen
3 days ago
agent-eval-loop — krit03w/krit-engineer-skills