On-Call, Toil and SRE Teams

Keep operational work bounded and shrinking so engineers spend most of their time on engineering, and build the team practices that sustain that.

Sustainable On-Call and Operational Load

On-call works when pages are rare, meaningful and manageable, and the rest of the team's time goes to engineering that makes pages rarer.

Ops work ≤ 50% of the team's time (on-call, tickets, toil, manual operations). The rest is engineering. When ops exceeds this, redirect the overflow to the development team or pause launches until it's back under control.
On-call ≤ 25% of any individual's time, which implies a rotation of roughly 8 people at one site, or 6 per site across two sites for 24/7 primary + secondary coverage.
At most ~2 incidents per 12-hour shift on average. Each real incident (handling, cleanup, postmortem) takes hours; more than that and quality drops. Zero for long stretches means skills atrophy — run drills.
Response time matches the SLO: a tight SLO needs minutes-level acknowledgment; a loose one can tolerate longer.
Multi-site rotations ("follow the sun") avoid night shifts but add handoff and communication cost.
Compensate out-of-hours on-call (time off or pay) and cap it so it stays bounded.