Alert Rules

Purpose

Design alert rules following the RED/USE method with severity classification, threshold tuning, notification routing, and escalation paths. Ensure every alert is actionable, documented with a runbook, and tuned to balance detection speed against fatigue.

Framework and Methodology

Alerting Philosophy

Four principles guide alert rule design:

1. Actionability — if no action can be taken, it belongs on a dashboard, not as an alert.
2. Signal over noise — every firing alert represents a real issue requiring human intervention.
3. Tiered response — severity dictates speed of response, not importance of the metric.
4. Continuous tuning — thresholds degrade over time as systems evolve; review monthly.

RED Method (Applications)

Rate, Errors, Duration — the three golden signals for application monitoring.