SkillOpt — Controlled Skill Optimization for Hermes Agent

Optimize any Hermes skill document using a rigorous, methodology-driven pipeline inspired by Microsoft Research's SkillOpt paper (Yifan Yang et al., 2025). The core insight: evaluate skill changes by measuring task execution, not by reading the skill text.

When to Use This Skill

• A skill exists but its performance is inconsistent — sometimes it works, sometimes it doesn't
• You want to improve a skill but aren't sure which changes will actually help
• You've proposed edits to a skill and want to validate them before deploying
• You maintain multiple skills and want a repeatable process for quality improvement
• You're curious whether a skill change actually improved anything (the surface-plausibility trap)
• You've just created a new skill and want to pre-optimize it before publication — catch discoverability and structural issues (buried principles, missing phase mappings, unanchored procedures) before any user encounters them. The edits are prominence and navigation fixes, not performance patches. This session's spec-driven-development run is the worked example: all baseline rollouts passed, but Epoch 1 found core principles buried in a reference file and a Quick Start that lacked reference-to-phase cross-links.

SkillOpt is designed for skills with measurable task outcomes. For creative skills (image generation, writing) where "correctness" is subjective, the methodology still works but the validation criteria need thoughtful definition. Crucially, the acceptance criteria themselves may need user iteration — the first set of criteria you propose may not match what the user considers "correct" for creative output. Propose criteria alongside your edit proposals and get user sign-off before running validation. This session's image-magnus919 run is the worked example: the initial validation criteria included a 30-word SUBJECT limit that the user correctly rejected as contradicting the rich, complex subjects that produce the best covers.

How It Works

SkillOpt treats a skill document like a parameter vector in text-space. Instead of gradient descent, it uses a six-phase kanban pipeline that runs on your Hermes Agent's existing infrastructure: