Graceful Degradation

Pain Signals — You Need This Pattern When:

• The primary model goes down and the entire feature stops working
• LLM latency spikes cause timeouts and users see error pages
• API costs are unpredictable and occasionally spike beyond budget
• All inputs — trivial and complex — go to the same expensive model
• There is no defined behaviour for "what happens when the AI fails"
• A simple keyword query hits the LLM when a database lookup would suffice

Core Principle

An LLM call is an external service call with all the same failure modes: latency spikes, outages, rate limits, cost overruns. Production systems must degrade gracefully, not catastrophically.

Define degradation levels upfront — what the system does at each level of failure.