flaky-test-diagnosis

Installation
SKILL.md

Flaky Test Diagnosis

Use this skill to turn a non-deterministic test into a deterministic one by finding why it sometimes fails, not by retrying it until it passes. The recurring failure mode it prevents is "fix" by sleep increase, retry annotation, or disabling — all of which hide the defect (which is sometimes in the code, not the test) and erode trust in the suite.

The governing rule: a flaky test is a bug report — either the test races/leaks, or the code does. Find which, then remove the non-determinism at its source.

WORKFLOW SKILL. INVOKES: build/run tooling to reproduce (repeat runs, shuffled order, parallel, sanitizers), read-only inspection of the test and code under test. Apply the deterministic-wait patterns of the project's test framework to express the fix, and hand a confirmed data race or lifetime bug to a concurrency/object-lifetime review rather than masking it. FOR SINGLE OPERATIONS: classify one observed flake against the source table and name the fix.

Scope

  • Use this skill for: reproducing and root-causing an intermittently failing test and making it deterministic, or escalating to a real code defect when the test correctly caught a race/leak.
  • Out of scope: authoring a new test from scratch (a test-framework authoring concern), judging whether a consistently-passing test is a good test (a separate test-quality-judgement concern), and fixing the underlying concurrency/lifetime bug itself once identified (a concurrency / object-lifetime review concern). Name the boundary when a finding depends on it; route the work accordingly.

First: reproduce deterministically

Do not trust a single green run. Increase the failure rate until you can observe it:

Installs
2
GitHub Stars
2
First Seen
Jun 28, 2026