os-eval-runner
Installation
SKILL.md
Skill Improvement Evaluator
Stateless evaluation engine that scores and gates skill improvement iterations using headless Python evaluation scripts.
Ownership Boundary (Critical)
What os-eval-runner owns (permanent, version-controlled with this skill)
- Scoring scripts:
./scripts/evaluate.py,./scripts/eval_runner.py - Scaffold script:
./scripts/init_autoresearch.py - Templates:
./assets/templates/autoresearch/(program, evals, results, proposer prompt)
What lives with the target (deployed per experiment)
All experiment state deploys alongside the target (e.g. <experiment-dir>/references/program.md, <experiment-dir>/evals/evals.json, <experiment-dir>/evals/results.tsv). You MUST read the spec from <experiment-dir>/references/program.md and NOT fall back to engine-local config templates.