Capacity, Overload and Reliable System Design

Design with real numbers, survive the loss of your largest failure domain, and make every component degrade gracefully instead of collapsing under load.

Overload and Cascading Failures

A cascading failure is one that grows by positive feedback: one part fails, its load moves to the rest, which then fails too. Most are driven by overload, so the defense is to make every service degrade gracefully instead of falling over.

How cascades happen

• Server overload → slower responses → more concurrent requests → more memory/CPU → slower still.
• Resource exhaustion: CPU (queues grow), memory (GC thrash, OOM kills), threads, file descriptors, connection pools.
• Retry amplification: every layer retries, multiplying load on the struggling backend.
• Crash loops / health-check churn: overloaded tasks fail health checks, get removed, pushing more load onto survivors.
• Slow startup / cold caches: restarted tasks can't take full load yet.
• Triggers: traffic spikes, a lost cluster/zone, bad releases, cache flushes, batch jobs, a planned change that reduces capacity.

Defenses (design time)