Prompt-Injection Defense

Concept of the skill

Data-vs-directive collapse: every token in the LLM context window contributes to next-token prediction, and the model has NO reliable mechanism to distinguish "directives from the application developer" from "directives in a document the application happens to have loaded." Four-cell threat matrix: direct injection (attacker is the user) / indirect injection (attacker controls retrieved content the agent reads) / action-trigger (attack causes the agent to invoke a destructive tool) / exfiltration (attack causes the agent to leak data via a tool call or rendered output). Attackers will use obfuscated and multimodal carriers (e.g., CSS-hidden prompts, Base64, multilingual attacks, HTML comments, markdown injections, and fake tool commands) to bypass simple filters. Every input surface is a potential vector: user input, RAG retrieval, tool result, attached document, multimodal content, subagent output. Defense is architectural containment — a layered stack from weak (input filtering, system-prompt warnings) to strong (capability constraint, dual-LLM pattern, planning/execution separation, human-in-the-loop, principle of least authority).

Replaces "build a smarter fence around the model" with "engineer the system so the model's mistakes do not matter." Without this framing, every defense attempt focuses on the model itself — patching the model, improving the system prompt, blocklisting attack strings — and produces partial reductions in attack success rate that never reach zero, because the vulnerability is structural to how transformer-based language models consume their input, not a bug to fix. The discipline accepts the vulnerability and contains its blast radius via architecture: capability constraint, origin tracking, dual-LLM pattern, planning/execution separation, human-in-the-loop confirmation for irreversible actions, principle of least authority on the tools the agent can call. The model can be tricked; the runtime must not be.

Distinct from security-fundamentals, which owns the general security framing (threat modeling, Saltzer-Schroeder principles, authn/authz, defense in depth, OWASP Top 10) — prompt-injection-defense is the LLM-specific specialization (prompt injection is OWASP LLM01, one row in the LLM Top 10). Distinct from tool-call-flow, which owns the protocol cycle by which a model invokes a tool (request/response shape, error handling, parallel calls) — prompt-injection-defense owns the security property that cycle must preserve when any message carries untrusted content. Distinct from guardrails, which owns model behavior policy and refusal boundaries — prompt-injection-defense owns the threat where untrusted content gets the model to perform actions it was not supposed to take. Distinct from owasp-security, which owns SQL injection, XSS, CSRF, and general application hardening — prompt-injection-defense owns the threat that arrives over correct HTTP and is still harmful because the model interprets it as a command. Prompt injection defense is to LLM-integrated systems what blast walls are to fuel depots — you cannot prevent the fuel from being flammable (the structural property), so you do not try; you build the walls so that an ignition contains itself, the radius is bounded, and the rest of the depot survives. The walls are the architectural defense; the model's susceptibility is the fuel's flammability — a property of its physics, not a bug to fix. The wrong mental model is that prompt injection is a bug in the model that better training, better system prompts, or content filters will fix. It is not. It is a structural property of how transformer-based language models consume their context: every token contributes to next-token prediction, and the model has no reliable mechanism to distinguish "directives from the application developer" from "directives in a document." A content filter that blocks one canonical attack phrase does not stop the broader class — paraphrasing, encoding, indirection, obfuscated and multimodal carriers (Base64, CSS-hidden instructions, HTML comments, fake tool commands, markdown injections), and the underlying structural property all combine to make the attack a moving target. The defenses that work are architectural (limit what tools the model exposed to untrusted content can call, separate planning from execution, require human confirmation for irreversible actions), not behavioral.

Coverage

The architectural discipline of defending language-model-integrated systems against the attack class in which untrusted content causes the model to follow attacker-controlled directives. Covers the data-vs-directive collapse that makes this attack structural rather than incidental, the direct/indirect/action-trigger/exfiltration taxonomy, the injection surfaces (user input, RAG retrieval, tool result, attached document, multimodal image content, subagent output), why content filters and improved system prompts do not solve the class, and the defense-in-depth measures that do (capability constraint, origin tracking, dual-LLM pattern, planning/execution separation, human-in-the-loop confirmation, principle of least authority).

Philosophy of the skill

This attack class is not a bug. It is a property of how transformer-based language models consume their context. Every token in the context window contributes to the next-token prediction, and the model has no reliable mechanism to distinguish "directives from the application developer" from "directives written by an attacker in a document the application happens to have loaded." Treating it as a bug to fix — by patching the model or improving the system prompt — buys partial reductions in attack success rate but never reaches zero.

The discipline of defense, therefore, is not to eliminate the vulnerability. It is to ensure that successful compromise does not translate to consequential action. The model can be tricked; the runtime must not be. The defenses that work are architectural: limit what tools the model exposed to untrusted content can call, separate the agent that reads untrusted content from the agent (or code) that takes action, require human confirmation for high-impact operations regardless of model intent, and track the provenance of every byte in the context window so that low-trust content cannot route to high-authority execution paths.

The wrong mental model is "build a smart fence around the model." The right mental model is "engineer the system so the model's mistakes don't matter."