Flaky tests

A flaky test is a real nondeterminism, in the test or in the product. Measure it, name the source, remove it, and prove the fix with the same measurement. Never hide it behind a retry.

Process

1. Measure before touching anything. Run the single test or file many times with the bundled script from this skill's folder, and keep the logs of failing runs:

   node scripts/repeat.mjs --runs 50 -- <test command for one file>
   node scripts/repeat.mjs --runs 200 --until-fail -- <command>
   node scripts/repeat.mjs --runs 30 --parallel 4 -- <command>   # add contention