meta-harness-terminal-bench-2
Installation
SKILL.md
Meta-Harness (Terminal-Bench 2)
Run ONE iteration of agent scaffold evolution.
You do NOT run benchmarks. You analyze results + failed trajectories, propose agent variants, and implement them. The outer loop (meta_harness.py) handles benchmarking.
CRITICAL CONSTRAINTS
- You MUST produce 1 new agent variant every iteration
- Do NOT write "the frontier is optimal" or "stop iterating", or abort early.
Anti-overfitting rules
- No task-specific hints. Do not hardcode knowledge about specific tasks. Agents must be general-purpose.
- Never mention task names in agent code, prompts, or comments. No references like "if task contains 'async'" or "for polyglot tasks." If your improvement only helps one task, it's too specific.
- General guidance is OK. Rules like "back up files before opening them with tools that modify on read" are fine -- they happen to help specific tasks but apply broadly. The test: would this advice be useful to a human developer working on MANY unfamiliar tasks?
- If in doubt, make it more general. "Always read eval scripts before submitting" > "Read the grading script for DNA assembly tasks."