mutation-test-review
Mutation test review
This skill judges whether tests would catch real bugs. Line coverage only proves that code ran. Mutation testing injects small deliberate faults (flip < to <=, return Default::default(), drop a call) and re-runs the tests. A mutant that no test notices marks a behaviour the tests never pin down. That signal is what makes this skill worth running, especially on AI-written tests that execute plenty of code while asserting very little.
A mutation score on its own is weak evidence. A tool can report 100% because every mutant in the interesting functions failed to compile, or because the run silently skipped a third of the test files, or because a third of the mutants timed out. The value of this review comes from checking that the numbers mean something and from the triage that follows, so spend your effort there.
The user invoked this skill explicitly, and the argument (if any) is $ARGUMENTS.
Outcome vocabulary
Use these five classes everywhere (in the summary table, the triage, and the prose) so reports stay comparable across tools.
| Class | Meaning | cargo-mutants | Stryker |
|---|---|---|---|
| killed | At least one test failed, so the fault was detected. | caught |
Killed |
| survived | Tests ran the mutated code and still passed, which points to an assertion gap. | missed |
Survived |
| no coverage | No test executed the mutated code at all, which points to a missing test. | not reported natively (see the Rust reference) | NoCoverage |
| timed out | The mutant made tests hang, usually an infinite loop. It counts as detected. | timeout |
Timeout |
| error | The mutant did not compile or crashed the runner. It is excluded from the score. | unviable |
CompileError, RuntimeError |