agent-output-audit
Installation
SKILL.md
Agent Output Audit
You are the independent evaluator. Answer one question — "Did the implementing agent actually do what task_NN.md says it did?" — from files, public behavior, tests, and CI. A self-report is not evidence. (Whether a real user can succeed at the product is qa-execution; run both on a Compozy slug and keep their outputs separate.)
Step 1: Discover the Repository Verification Contract
- Read root instructions, repository docs, and CI/build files before running commands.
- Run
python3 scripts/discover-project-contract.py --root .to surface candidate install/verify/build/test/lint/start commands and E2E signals. - Prefer repository-defined umbrella commands (
make verify,just verify, CI entrypoints) over language defaults. When discovery surfaces more than one plausible gate or mixes ecosystems, readreferences/project-signals.mdbefore choosing, and state the tie-breaker. - Read
references/e2e-coverage.mdbefore classifying any flow's coverage. - Resolve the audit artifact directory: the
audit-output-pathargument if given, else repository conventions, else/tmp/agent-output-audit-<slug>. Create itsaudit/subdirectory; store all bugs and reports under<audit-output-path>/audit/. - Detect Compozy mode. If
.compozy/tasks/<slug>/exists, record the slug and:- Read
state.yamlread-only —scripts/update-state.pyowns its mutation per the cy-codex-loop contract. - Read
_techspec.md(deliverable source of truth) and_tasks.md(task roster) when present. - List every
task_NN.mdand capture its frontmatterstatus:(pending|in_progress|completed). When frontmatter disagrees withstate.yaml, frontmatter is the source of truth. - Note the memory slot
.compozy/tasks/<slug>/memory/qa-execution.md— Step 5 writes it before any status flip.
- Read