Compare Trials

Use this skill for analysis over graded JSONL outputs, typically after:

agent-eval-harness eval '{"mode":"grade",...}' > graded.jsonl

Default path

For normal baseline-vs-challenger comparisons, use:

agent-eval-harness eval '{"mode":"compare",...}'

Use custom scripts only when suite-specific analysis is required.

Custom analysis use cases