short-form-eval
Installation
SKILL.md
Short-Form Eval — Orchestrator
Feedback-loop skill — closes the brief → publish → score → pattern-log loop. The gap-gate consumes its outputs to decide what the stack should learn next.
Core Question: "Did the brief survive contact with the platform — and what's the signal-bearing pattern this cycle adds to the log?"
Critical Gates — Read First
Non-negotiable constraints before dispatching any agent:
- Provisional rubric, not locked.
references/rubric.mdships atversion: 0.1, status: provisional. Mandatory revision after cycle 2-3 against real variance. Per-cycle rubric drift is expected — encode the change in the artifact, don't smuggle it into the rubric file silently. - Cycle 1 weighting is 70% observation / 30% scoring. Single calibration pair would overfit a locked rubric. First cycle leans toward describing what you saw; later cycles harden scoring as variance accumulates.
- Both brief and reference catalog must exist. No platform-intel reference → BLOCKED. No brief → BLOCKED. The eval scores a fidelity claim against known patterns; missing either side reduces the run to vibes.
- No fabricated metrics. Every engagement number, completion rate, save/share count, and sample-size claim cites the URL or panel screenshot it came from. Critic rubric #1 fails the artifact otherwise.
- Pattern-log entries are atomic. One cycle = one pattern-log entry block in the report. The block has a fixed shape (claim, evidence, refutability, expiry) so future cycles can diff. Free-form prose patterns are unusable downstream.