testing
Testing
Coverage is a signal, not the goal. Choose evidence that establishes required production behavior and material risks, including unusual states, misuse, partial failure, and recovery. Keep the agreed quality bar fixed.
Verification Cost And Risk
Before adding or running a test, name the behavior or evidence gap, inspect existing coverage and still-valid results, then choose the lowest-total-cost sufficient check. Count authoring, setup, execution, agent output/diagnosis, maintenance, and likely rework from delayed feedback; do not invent precise token estimates.
- Prevent failures first. Reuse proven code, simplify branches/state, and use types and data constraints. Static checks and review answer appropriate questions, but cannot prove runtime behavior or replace required tests.
- Limit claims to exercised behavior. Faithful external fakes support executed logic and caller-interaction assertions, but cannot prove a replaced outcome-owning component's behavior. A mocked save does not prove durable state; a captured request proves shape/intent, not transport or remote delivery. Separate tests against independent doubles do not establish seam compatibility. Shared types/fixtures prove only contracts actually checked. Reuse sufficient valid contract/integration evidence, otherwise choose the cheapest sufficient seam check or report the gap.
- Choose timing by late-discovery cost. Check risky contracts, authorization, money, and data integrity before dependent work compounds uncertainty. Otherwise defer to a named trigger such as integration completion or phase acceptance. Deferred checks remain pending, with a named owner when work is delegated; deferral is not a waiver or pass.
- Batch and reuse. Share setup in one invocation and rerun only invalidated scope. Do not remove regression coverage to shrink a run. Select affected tests only when dependencies are understood; investigate uncertain impact and broaden where unresolved.
- Bound agent overhead. Retain concise streaming progress, commands, scope, and results; inspect failure details as needed. Share one evidence record across workers rather than copying logs or duplicating runs.
- Automate repeated checks. Before running the same manual check a second time, add it to the project's automated tests when its outcome can be asserted stably, extending an existing test or journey before adding one; reuse coverage that already proves it. Skipping that needs a stated cost or judgment reason; a check that needs human judgment stays manual. Run deterministic steps as one batch and read the reported outcome instead of driving each step.
- Stop with sufficient evidence. Required behavior and material risks need current evidence and required checks must pass. Then stop adding tests, full-suite runs, or reassurance reviews. Whole-application verification needs a user/project requirement or unresolved cross-feature risk that narrower checks cannot cover. Cost limits never justify weaker assertions, a lower bar, or an incomplete check labelled passed; disclose gaps and reassess the plan.
Test Layers
Choose each test's layer by asking where the behavior breaks and which is the cheapest layer that can observe it; verify the behavior at the lowest such layer. Duplicate assertions across layers need distinct failure mechanisms. Prefer public APIs, service boundaries, database constraints, queue handlers, and integration seams over private helpers.