SemanticALLI: Caching Reasoning, Not Just Responses, in Agentic Systems

This skill teaches Claude to design and implement stage-decomposed caching for agentic AI pipelines. The core insight from SemanticALLI is that even when users never repeat the same query, the pipeline's internal reasoning often converges on identical structured intermediate representations (IRs). By decomposing a monolithic LLM call into discrete stages — each producing a cacheable structured artifact — you can bypass the majority of redundant LLM invocations. In evaluation, this approach lifts cache hit rates from ~39% (monolithic prompt caching) to ~83% (IR-level caching), cutting token consumption by 78% and reducing median cache-hit latency to 2.66ms.

When to Use This Skill

• When building an agentic pipeline that chains multiple LLM calls (e.g., intent parsing -> data retrieval -> code generation -> output formatting) and you want to reduce cost and latency
• When the user says "my agent pipeline is too slow/expensive" and you need to identify redundant reasoning
• When designing a caching layer for an AI system that handles paraphrased versions of the same underlying request
• When the user asks to "add caching to my LLM agent" and naive prompt-level caching isn't sufficient
• When building analytics, dashboards, or code-generation agents where the same structured logic recurs across different natural-language inputs
• When implementing a multi-stage agent where early stages normalize intent and later stages synthesize output from that normalized form

Key Technique

The problem with monolithic caching: Traditional approaches cache at the boundary — hash the user prompt, check for an exact or semantic match, return the cached response. This fails when users phrase the same underlying request differently. "Show me monthly revenue trends" and "Plot revenue by month" have low lexical overlap but identical downstream logic. Monolithic caching caps around 38-40% hit rate because it cannot see past linguistic variance.

Stage decomposition and IR elevation: SemanticALLI decomposes inference into at least two stages: (1) Analytic Intent Resolution (AIR) — which normalizes a natural language query into a canonical structured IR containing metrics, dimensions, filters, temporal grain, and output primitives; and (2) Visualization/Synthesis (VS) — which converts that IR into executable output (code, charts, API calls). The IR becomes a first-class cacheable artifact. Because the IR is structured and stable under paraphrase, the VS stage achieves dramatically higher cache hit rates. Two completely different user prompts that resolve to the same IR will share VS cache entries.