short-form-eval

Installation
SKILL.md

Short-Form Eval — Orchestrator

Feedback-loop skill — closes the brief → publish → score → pattern-log loop. The gap-gate consumes its outputs to decide what the stack should learn next.

Core Question: "Did the brief survive contact with the platform — and what's the signal-bearing pattern this cycle adds to the log?"


Critical Gates — Read First

Non-negotiable constraints before dispatching any agent:

  1. Provisional rubric, not locked. references/rubric.md ships at version: 0.1, status: provisional. Mandatory revision after cycle 2-3 against real variance. Per-cycle rubric drift is expected — encode the change in the artifact, don't smuggle it into the rubric file silently.
  2. Cycle 1 weighting is 70% observation / 30% scoring. Single calibration pair would overfit a locked rubric. First cycle leans toward describing what you saw; later cycles harden scoring as variance accumulates.
  3. Both brief and reference catalog must exist. No platform-intel reference → BLOCKED. No brief → BLOCKED. The eval scores a fidelity claim against known patterns; missing either side reduces the run to vibes.
  4. No fabricated metrics. Every engagement number, completion rate, save/share count, and sample-size claim cites the URL or panel screenshot it came from. Critic rubric #1 fails the artifact otherwise.
  5. Pattern-log entries are atomic. One cycle = one pattern-log entry block in the report. The block has a fixed shape (claim, evidence, refutability, expiry) so future cycles can diff. Free-form prose patterns are unusable downstream.

Related skills

More from hungv47/research-skills

Installs
1
GitHub Stars
1
First Seen
4 days ago