gspdev-eval-changes
Installation
SKILL.md
- Change intent achieved? Did the PR/commit land what it set out to do?
- Skill intent coherent? Does the skill body deliver on its description? (Catches drift between SKILL.md frontmatter
description:and the body — a separate failure mode from the change-intent question.) - Substance preserved or improved? For modifications: did the change avoid sacrificing what the agent needs? For additions: does the new content fit the skill's existing contracts (constraints, output specs, quality bars)?
Examples of changes this covers:
- Adding a new step to a methodology
- Adding a new sibling reference file
- Modifying a rule, constraint, or rubric
- Changing the agent's role or execution mode
- Trimming verbose prose
- Extracting content to siblings
- Renaming inputs/outputs
- Adding a new SKILL.md (skill intent check from scratch)
A change that achieves its goal (Dim 1 = PASS) can still drift the skill away from its stated purpose (Dim 2 = FAIL) if it adds behavior the description doesn't reflect, or removes a step the description promised. A coherent intent + description (Dims 1-2 = PASS) can still break an output spec (Dim 6 = FAIL). All three gates matter.
Spawns one general-purpose evaluator agent per modified file in parallel. Each evaluator compares BEFORE (if any) and AFTER against the parent SKILL.md using a fixed 8-dimension rubric, returns PASS / CONCERNS / FAIL with quoted findings and surgical recommendations.