Production Failure Modes & the Inference-Stack Tradeoff Map

This is the capstone. The four sibling skills each own one slice of the stack — harness-and-context-engineering (the agent loop, prompts, tools, context packing), rag-architecture (retrieval, grounding, indexing), inference-caching-and-kv (serving, prefix/KV cache, throughput), quantization-and-model-compression (weight/activation precision, model size). This skill ties them together at the layer where they all meet: what breaks in production, and what every fix costs you elsewhere.

The thesis: LLM systems fail silently

A traditional service fails loud — a 500, a stack trace, a thrown exception, an alert. An LLM system mostly fails quiet. The HTTP 200 still comes back. The JSON still parses. The agent still finishes. The answer is just worse — subtly wrong, stale, refused, hallucinated, or off-policy — and nothing in your exception tracker fires.

Three consequences drive everything below: