Skill Evaluator

IRON LAW: Evaluate a skill against real-use failure modes, not against how convincing its instructions sound.

Purpose

Assess whether a skill is discoverable, concise, executable, verifiable, and robust under realistic agent behavior. Turn observed failures into concrete workflow gates, scripts, references, or eval cases.

Operating Rules

• Treat a skill as a reusable capability module, not a long prompt.
• Prefer raw evidence: user prompts, agent transcripts, diffs, output artifacts, validation logs, and failed runs.
• Do not pass your diagnosis or intended fix into forward-testing prompts; pass the skill and realistic user request.
• Do not grade success by output polish alone. Check whether the agent followed the required process and left verifiable evidence.
• Do not require artificial success counts. Require process evidence that makes low-output results trustworthy.
• Keep recommendations specific: name the file, rule, script, eval case, or validator check that should change.
• If the user asks you to modify the evaluated skill, apply the smallest change that blocks the observed failure mode.

Workflow