test-writing

Test behavior through the widest boundary that stays hermetic and cheap to set up.

You are usually the one writing these tests, and you have specific habits that produce green suites with no protective value. Correct them first; the craft guidance below is downstream of that.

Habits to correct

• Writing the test from the code you just wrote. Reading the implementation and asserting what it does bakes in its bugs. Derive expected values from the requirement and compute them by hand. If you cannot state the expected answer without running the code, you do not understand the behavior well enough to test it yet.
• Copying actual output into expected. Running the test, seeing it fail, and pasting the actual value produces a recording, not a test. Same for widening a tolerance until it passes.
• Mirroring code structure. One test file per source file and one test per public method is a unit-of-code suite. Test units of behavior; how many classes implement one is irrelevant.
• Mocking by default. Mocks dominate training data and make things pass locally. If a mock is the only way to test something, the design is the problem, not the test.
• Repairing the test instead of the code. A failing test is a hypothesis about the code until proven otherwise. Never weaken an assertion, add a mock, or skip a test to reach green. Changing a test and the code it covers in one commit needs a stated reason why both were wrong.
• Mutating the environment to get green. Seeding a row by hand, flipping a flag, restarting a service, or draining a queue is the deployment-tier version of weakening an assertion — it passes, nothing is fixed, and the shared state has drifted for everyone else.
• Debugging against a shared environment by trial and error. A red run there is ambiguous by construction: your change, someone else's, stale data, or a broken env. Reproduce it locally in a hermetic test first; if you cannot, say so rather than guessing.
• Blind-accepting snapshots. Running the update mode without reading the diff turns the expect-test loop into an auto-approval loop. Treat an unreviewed snapshot update as an untested change.
• Generating volume. Cheap tests become near-duplicates that break together and bury the one real failure. Coverage percentage and test count are not goals.
• Making tests "robust." try/except, conditionals, retries, and sleeps inside a test make it unable to fail. Let it fail loudly and precisely.
• Testing the framework. Asserting that the ORM saves or the stdlib sorts tests someone else's code.