Incident Response

An incident is not a debugging session with an audience — it is a distinct mode with its own goals. The instinct of a good engineer is to understand the problem before acting; the discipline of incident response is to reduce customer harm before you understand anything at all. Those two goals conflict, and the second one wins until stated otherwise.

The single biggest failure mode in live incidents is not technical, it is organizational: nobody is in charge, three people fix the same thing three different ways, and no one can say what changed. Fixing that costs nothing and saves the most time.

Mitigate first, understand second, and let one person own the decisions.

For role definitions, a severity matrix, and copy-paste status-update templates, read references/incident-roles.md.

1. Name an incident commander in the first five minutes