harbor-benchmarks
Installation
SKILL.md
Harbor Benchmarks
Build a bounded mathematical experiment with a hidden, replayable verifier. Jacobian availability is an experimental condition, not a prerequisite for task admission. Task correctness depends on the submitted mathematics, not the agent's tool choices or transcript.
Choose the artifact
- Put meaningful agent-facing mathematical tasks with replayable outcomes in Harbor, including capability gaps and tool-free baselines.
- Put provider/environment feasibility spikes under
tests/fixtures/providers/. - Keep public known-answer regressions and human-reviewed proof corpora separate from held-out comparative scoring.
For submission design, mathematical predicates, scoring, or verifier repairs,
use verifier-evaluations. Ordinary packaging and execution do not require that
skill. A task's instruction and schema must expose its complete protocol while
keeping solutions, Oracle fixtures, verifier code, and host caches hidden.