AI Red Teaming Skill

Goal

Proactively attack your own AI/agent system to discover failures before an adversary does. This is the offensive complement to the defensive skills: prompt-injection-defense.skill builds the layered controls and ai-safety-guardrails.skill builds the rails — this skill tries to break both. Red teaming is threat-led: it designs scenarios from a model of who would attack and why, executes them as concrete attacks, scores the attack-success-rate (ASR), and gates CI on it so regressions are caught automatically. Stated honestly: red teaming reduces risk by finding failures, but it can never prove safety — absence of a successful attack in your suite is not evidence the system is safe.

Steps

1. Design threat-led scenarios mapped to OWASP LLM Top 10

   • Start from threat actors and goals (insider, untrusted external content author, compromised tool/MCP server, end user seeking disallowed output), not from a generic checklist. Reuse the surface enumeration from threat-model.skill rather than re-deriving it.
   • Map each scenario to the relevant OWASP LLM Top 10 category (e.g. LLM01 Prompt Injection, LLM02 Sensitive Information Disclosure, LLM06 Excessive Agency, LLM07 System Prompt Leakage) so coverage is auditable and gaps are visible.

2. Build the attack technique library

   • Direct injection: malicious instructions in the user's own prompt.
   • Indirect injection: instructions hidden in ingested content — RAG documents, web fetches, file contents, tool output, tool metadata, image alt-text.
   • Jailbreaks: persona/role-play, obfuscation, encoding, and refusal-bypass prompts targeting the safety policy.
   • Multi-turn / crescendo: benign-looking openers that escalate across turns to a disallowed goal that a single-turn filter would miss.
   • Data exfiltration: weaponizing a benign tool (link unfurl, image fetch, URL parameter, DNS) as a covert channel for private data the agent holds.
   • Tool abuse / excessive agency: coercing the agent into unauthorized, irreversible, or out-of-scope tool calls.
   • Training-data extraction: probing for memorized PII, secrets, or proprietary text in model outputs.