LLM Caching

Cut LLM costs and latency with exact match, semantic, and provider-side caching layers.

When to Use This Skill

Use this skill when:

• The same or similar queries are asked repeatedly (FAQ bots, support tools)
• LLM API costs are growing and you need immediate savings
• Serving high request volumes where repeated queries cause bottlenecks
• Implementing prompt caching for long system prompts (Anthropic/OpenAI)
• Building offline-capable AI features that need response persistence

Caching Layers

Cache examples are incomplete until authorization/data boundaries are included. Partition by tenant/user access scope and include model revision, system/developer prompts, conversation/tool context, parameters, locale and source/policy version in keys or eligibility filters. Never serve a privileged cached answer across users merely because queries are similar. Hashing prompts is not encryption: restrict/encrypt stored queries/results, honor deletion/retention and invalidate when permissions/data change. Avoid semantic reuse for sensitive or time-dependent answers unless evaluated false-hit risk is acceptable. Similarity thresholds/latencies/savings are examples, not calibrated guarantees. Cache warming is paid generation: bound concurrency, requests and budget before running.