Cached System Instructions (KV-Cache Prefix Architecture)

Overview

In Large Language Models, processing input tokens requires running dense matrix multiplications across the entire input sequence on every turn. In a 50-turn agent session with a 4,000-token system prompt, the provider re-processes the exact same 4,000 tokens 50 separate times ($200,000\text{ redundant token computations}$).

Prompt Caching (Anthropic Prompt Caching, OpenAI Automatic Prefix Caching, DeepSeek Context Caching) saves the transformer's Key-Value (KV) Activation States in GPU memory. Re-sending an identical prefix reuses the precomputed KV cache, cutting Time-to-First-Token (TTFT) latency by 80% and applying an automatic 90% cost discount.

The Hierarchical System Caching Protocol organizes instructions to guarantee uninterrupted prefix matching.

Uncached Re-computation vs. KV-Cache Hit