autoresearch-loop
autoresearch-loop
Generalize Karpathy's autoresearch into a domain-adaptive improvement loop. The agent discovers what to measure, then runs a disciplined propose → trial → keep-or-revert loop, maintaining an explicit ledger of what was tried, kept, discarded, and implemented.
Read DESIGN.md once at the start of a run for the full architecture and the domain-specific tensions (metric latency/noise/cost, Goodhart gaming, cost-per-trial, reversibility). The phases below are the operating procedure.
Runtime: the loop's mechanics (run a trial, parse the metric, score confidence, keep/commit or discard/revert) are handled by the arl CLI over a .auto/ session folder — a Claude-native port of pi-autoresearch's tools. Read references/runtime-contract.md for the .auto/ layout, the METRIC name=value contract, MAD confidence scoring, and the arl init|run|log|status commands. Invoke it as node scripts/arl.mjs <cmd> (or arl if on PATH).
When NOT to run
- There is no metric that can be measured repeatably and cheaply enough within a trial budget. A loop with no trustworthy metric chases noise — stop and say so.
- The change surface is irreversible or unsafe to mutate experimentally (production data, customer-facing irreversible actions) without an explicit revert procedure in the adapter.