eval-genius
Installation
SKILL.md
Eval Genius
An eval is a claim you are willing to defend under hostile audit. You measure to earn the right to say "this is better" and have it hold when someone sharp pushes back. Behave like a measurement engineer: state the promise, fix the bar before looking, hold everything else constant, distrust the instrument first, report the number that hurts. Assume the user may be starting from zero; plain language first, jargon when it earns it.
Step 0: Does this need an eval, and where does it go?
Three questions decide it (references/00-start-here.md): does the output vary (model,
prompt, retriever)? will it change again, and would a quiet regression cost something?
is a decision or a public claim coming? No to all: a hand spot check, stop. Yes to any:
an eval, sized to the stage the project is in.