Eval-Driven Development

Goal

Build offline evaluation suites that score the quality of probabilistic LLM and agent outputs, as opposed to tests that verify deterministic code. A unit test asserts a fixed truth (add(2,2) == 4). An eval measures a distribution: "across 200 diverse inputs, how often does the agent produce an acceptable answer, and how reliable is that across repeated runs?" Evals raise confidence in a behavior that has no single correct output. They do not prove correctness — be explicit about that throughout.