Production Reliability

Two questions, answered by two different bodies of practice. What is production actually doing right now, and who decides what happens when it stops? Duolingo's engineering blog is unusually candid on both — including where the approach has costs, where it failed, and where the honest version is that a marketing stunt paid for the infrastructure.

This skill is about signals from real production and the response when it fails. Designing, grading and constraining an LLM-driven test or migration agent is a different problem and belongs to the AI agent platform skill, not here.

Knowing what production is doing

• [[references/sampled-tracing-versus-request-identity]] — sampling finds systemic waste; one customer's problem needs always-on identity you can usually derive from logs you already write.
• [[references/find-code-that-runs-not-code-thats-referenced]] — static analysis cannot see code that is referenced but never executes, which is where concluded experiments leave residue.
• [[references/gate-the-release-on-internal-telemetry]] — internal use of a pre-release build gives a reliable stability signal within hours, if you attach a rule and an owner to it.
• [[references/keep-the-visual-record-your-tests-already-produce]] — your CI already renders every state on every device and locale, then deletes the evidence.
• [[references/make-reporting-free-and-aggregate-every-channel]] — capture the report in place with context attached, then pool every channel and look for what spikes.

When it breaks