Agent evals — proving the thing behaves

An agent's behaviour does not exist in its source. The code says what it is allowed to do; only a run says what it did. So the artifact under test is the execution record, and the suite is grown from production rather than authored up front.

Three claims follow, and they are what makes this different from testing ordinary code:

• You are testing reasoning, not code paths, so one granularity is never enough.
• Every natural-language input is unique, so the edge cases cannot be enumerated offline. Production is not only where you catch what you missed — it is where you discover what to test for.
• Traces become test cases. The suite grows from what actually happened.

1. Three primitives