Experiment arms

The experiment arm method turns a pile of ideas into measured, landed improvements. Each hypothesis gets its own small, revertible PR (an arm) that exists only to be measured. Arms run in parallel against a shared control that does the same work, and each one is judged against a keep bar you wrote down before looking. Winners get composed into a stack and landed at whatever autonomy level the repo allows. Losers become closed [negative result] PRs plus a ledger row, so no one has to try them again. The number you report at the end comes from real traffic after merge, not from the estimate.

Start from a Loop Card (templates/loop-card.md). If you don't have one yet, run find-the-loop first.

When to use / not use

Use it when:

• the metric has run-to-run noise and you can run the real system on demand (CI, a perf harness, an eval, a benchmark);
• there are several candidate changes, or one popular tip nobody has measured on your setup;
• the result must survive review by someone who will ask "compared to what?".

Don't use it when:

• one deterministic test settles the question. Write the test.
• the expected effect is smaller than the noise floor and you can't afford the n needed to see it. Pick a different instrument or a bigger lever.
• the system can't be run on demand. Observe production instead and say the evidence is weaker.

Roles