Production Debugging

Systematic production incident investigation. Mitigation comes before root cause — stop the bleeding first, then investigate why.

When to Use

• Something is broken or degraded in production right now
• Users are reporting errors or unexpected behavior in a live environment
• An alert fired and you need to understand what's happening
• You want to do a structured post-incident investigation

Iron Laws

1. Mitigate before you investigate. Reducing user impact takes priority over finding root cause.
2. Preserve evidence before changing anything. Capture logs, traces, and metrics snapshots before rolling back or restarting.
3. Never guess — instrument and observe. Form a hypothesis, then get evidence. Never propose a fix based on intuition alone.
4. Reproduce before fixing. If you cannot trigger the problem reliably, you cannot verify the fix.
5. Protect context window. All log output goes to a file. Never dump raw logs inline — pipe through grep/tail and show only relevant lines.
6. Clean before fixing. Run git restore . to remove all debug instrumentation before implementing the fix.