Runbook & Incident Response

Document operational procedures and incident response so on-call and ops can act quickly and consistently.

When to Activate

• User asks for "runbook", "incident response", "how do we deploy/rollback", or "ops procedures"
• Adding a new service or deploy process that needs documentation
• Post-incident: document what was done and what to do next time
• Setting up or improving on-call and escalation

Work Process

1. Identify procedure type — Deploy, rollback, failover, scaling, alert response, or incident flow. Gather from the user or from existing scripts and docs: what is done today, in what order, and with what tools.
2. Gather steps and commands — List actual commands, URLs, and tool paths. Test commands in a safe environment if possible (staging, dry-run). Note where credentials or secrets come from (vault, env); never put real secrets in the runbook.
3. Add structure — Purpose, prerequisites, numbered steps, expected outcome per step, failure handling ("if X fails, do Y or escalate to Z"), verification, rollback, and contacts.
4. Write for the on-call — Assume 3 a.m. and high stress. One action per step; exact commands; clear "success looks like this" and "if this fails, do that." Use code blocks for commands; specify shell and environment (e.g. production, staging).
5. Review and place — Put in repo (e.g. docs/runbooks/) or link from README/ops doc. Align with existing runbook style and tooling (PagerDuty, status page) if present. Keep contacts and escalation paths updated.