test-strength
Test strength
Execution is not protection. A strong test turns red when the guarded behavior is broken and exercises the layer where the claim lives.
1. Build the behavioral model
State the invariant, failure family, runtime boundary, and existing tests that claim protection.
For protocols, state machines, parsers, serializers, and event translation, write the dimensions before reading the implementation's branches. Derive them from each independent owner of behavior: specification or reference implementation, host/runtime semantics, and product contract. Include prior state, input or event, action or transition, negotiated modes or flags, relevant environment, and expected output plus post-state.
Enumerate every meaningful cell or partition it into documented equivalence classes. Mark impossible or excluded cells with the external rule that excludes them. Pairwise sampling is insufficient unless interactions between omitted dimensions are proved independent.
Complete when: the report contains the dimensions, values, exclusions, and the smallest wrong implementations the tests must reject. “Matrix run” without the table is incomplete.
2. Establish an independent oracle
Use a written specification, reference implementation, real producer or consumer, captured conformance vectors, or an independently written model. The production branch structure, constants copied from it, and expected values generated by the same helper are not independent.
For lifecycle events, validate fixture semantics against the real producer. Model each transition as pre-state + event → output + post-state. Do not reuse one mutable or contradictory fixture for press and release, connect and disconnect, or active and cleared state merely because the type accepts it.