evaluation-audit
Installation
SKILL.md
Evaluation Audit
Return one verdict for the audited artifact and the decision it serves. Launch and hold apply to an instrument before it is used: an eval, benchmark, harness, judge, or scoring change. Report and hold apply to a result before it informs a decision or reaches a reader.
- Launch: the instrument can carry the named decision at the named scope. Launch is a verdict, not authorization; the campaign or merge proceeds under its own approval.
- Report: the result can inform the decision or reach its reader at the stated strength, with the listed caveats.
- Hold: at least one claim fails, or is unverified in a way that could change the decision, and the table names what clears it.
State the decision and its scope first (a pilot, a full campaign, a merge, an internal decision, a customer report), because the same eval can be fit for one and not another. Give the verdict as a table with one row per readiness claim.
Synthetic example. Verdict: hold, for a full campaign.
| Claim | Evidence | Verdict | Condition to clear |
|---|---|---|---|
| Reference answers pass the grader | Oracle probe, 5 cases, runner at a1b2c3d: 5 of 5 pass |
Holds | None |
| Timeouts are excluded, not scored | runner/score.py:88 maps a timeout to score 0 |
Fails | Owner routes timeouts to the error record; rerun the null probe |
| Judge agrees with human reference labels | No validation results for the current judge prompt | Unverified | Validation on independent labels (evaluation-design) |
Row verdicts are holds, fails, or unverified. Unverified is not failed: keep a defect apart from evidence that was unavailable.