Agent Regression Evals — prove the change did not make it worse

Portability: 3 stdlib scripts, no APIs, no LLM calls. You run the agent; these grade, count and gate.

"Pass rate went from 82% to 79%" is not a finding until you know whether three points is outside the noise. Agents are non-deterministic, so one trial per case cannot tell a regression from a coin flip, and a final answer can be right while the path to it called a tool it never should have.

Workflow