leaderboard

Installation
SKILL.md

nurb leaderboard

Two generated surfaces, one editorial layer. evals/REPORT.md (audit tables) and site/benchmarks.html (the page behind nurb.dev/benchmarks) both regenerate mechanically from evals/submissions/; the verdict sentences and subscription labels live in evals/src/nurb_evals/site.py and are written by a person. Submission PRs are pure additions and merge freely; nothing reaches the public page until this skill runs. That gap is deliberate: it is where the sanity check and the verdicts happen, so publishing is an editorial act, not a side effect of merging.

Step 1: Find what is new

The runs added since the page was last regenerated:

git log -1 --format=%H -- site/benchmarks.html
git diff --stat <that-commit>..HEAD -- evals/submissions/

Each new directory under evals/submissions/ is one run: <harness>-<model>-<effort>-<hex>/ holding results.jsonl plus per-trial transcripts and part sources. If nothing is new, say so and stop.

Step 2: Sanity-check every new run

Work from evals/ (its own uv project; uv sync --locked first if the venv is stale). For each new run directory:

Installs
34
Repository
shpigford/nurb
GitHub Stars
116
First Seen
9 days ago
leaderboard — shpigford/nurb