LiteLLM Caching

Source: https://docs.litellm.ai/docs/caching/all_caches, https://docs.litellm.ai/docs/proxy/caching

Purpose

LiteLLM caches LLM responses to save costs and reduce latency. Identical requests return the cached ModelResponse instead of calling the LLM API. The default cache key is computed from model + messages + temperature + logit_bias. Semantic caching (Redis/Qdrant) uses embedding similarity for cache hits on semantically similar queries.

When to Use

• Repeated identical requests that shouldn't hit the LLM each time
• Cost reduction in agent loops where the same tool call repeats
• Multi-instance deployments needing shared cache (Redis, S3, GCS)
• Semantic caching for "similar enough" queries
• Persisting cache across restarts (Disk cache)

Basic Usage

Quick Start — In-Memory (SDK)