judge
Installation
SKILL.md
judge
Backs subagent output evaluation across all /shannon:dispatch* commands. Implements the two-stage meta-judge → LLM-as-judge pattern with structured YAML verdicts, hidden-threshold bias mitigation, and integration with Shannon's iron-rules dimensions.
Two-stage pipeline
Stage 1 — meta-judge (delegated to agents/meta-judge.md):
- Receives the original task, context, artifact type.
- Generates an evaluation specification YAML containing rubrics, checklists, scoring criteria, forbidden patterns, and evidence requirements.
- Output is a single block of YAML, written verbatim to
.shannon/state/rubric-<run-id>.yaml. - Pass-threshold is NEVER included in the YAML. Judges must remain blind to the threshold to prevent bias.
Stage 2 — judge (this skill):
- Receives the meta-judge YAML verbatim, the artifact under evaluation, and the original task.
- Applies the rubric mechanically — score each criterion with cited evidence.
- Emit a structured YAML verdict header at the START of the response.
- The orchestrator parses only the header, never reads the full report.