hillclimb-skill

Installation
SKILL.md

Hillclimb Skill

Improve a skill against its eval one round at a time: one change, one run, one verdict. Requires caliper (pipx install caliper-eval if missing) and python3. Commands, files, the cause list and the report: REFERENCE.md.

Three terms carry the method:

  • Train / held-out: the tasks are split once. You read train failures; held-out tasks reach you as scores only. A change that lifts train alone has learned the tasks, not the job.
  • Noise floor: how far a split's score moves between two runs of the same skill. A gain inside it is not a gain.
  • Keep / revert: every round ends in one of the two, decided by the verdict table.

The spec and every results JSON are the answer key. Reach them only through climb.py and caliper compare --format json, and send each caliper run's output to a log file.

Entry point

/hillclimb-skill [path]: optional path to a SKILL.md; without one, use the SKILL.md in the cwd. Its eval is the *.eval.yaml beside it.

  • No eval: stop and suggest grill-skill, which builds one.
  • Fewer than 8 tasks: stop. There are too few to hold any out; suggest adding tasks with grill-skill.
  • A git-sourced skill: stop. Only a path source can be edited.
Installs
3
GitHub Stars
235
First Seen
1 day ago