QEDGen Auditor Benchmark

Measure the portable auditor skill without leaking labeled findings into audit workers. Require a venue-owned benchmark directory; never assume a particular home directory or agent harness.

Inputs

Require:

• canonical auditor skill directory;
• a corpus manifest conforming to schemas/corpus-manifest.schema.json, with repository, audited commit, program root, runtime, setup/test commands, sanitization rules, labeled findings, optional domain expectations, and a required entry-level difficulty;
• explicit benchmark output directory;
• audit profile and run count;
• exact discovery and judge model identifiers when the venue permits selection.