llm-red-team

Installation
SKILL.md

Skill: LLM Red Team

Supplementary Files:

  • payloads.md — Prompt injection payloads (direct/indirect/encoded), jailbreak corpora (DAN, roleplay, many-shot, encoding), model-extraction probes, RAG-poisoning document templates, membership-inference scripts, agentic tool-abuse payloads, safety-filter bypass techniques (multi-language, tokenizer tricks, homoglyphs), hallucination induction patterns, model DoS vectors, perimeter recon (model fingerprinting), reporting templates, and defense-evasion obfuscation patterns
  • test-cases.md — 12 structured test cases (TC-LR-001 .. TC-LR-012) covering recon, direct injection, indirect injection, jailbreak, extraction, RAG poisoning, agentic abuse, safety bypass, hallucination, DoS, defense bypass, and full red-team report
  • guides/llm-red-team-playbook.md — End-to-end red-team playbook (pre-flight authorization, six-phase methodology, tool deep-dives on promptfoo/garak/PyRIT/PurpleLlama, multi-turn attack-chain design, evidence collection, OWASP LLM Top 10 mapping, and the purple-team feedback loop)
  • guides/llm-jailbreak-arsenal-playbook.md — State-of-the-art jailbreak arsenal (DAN/STAN/AIM persona patterns, Anthropic many-shot jailbreaking, Microsoft Crescendo multi-turn escalation, PAIR/TAP automated adversarial rewriting, GCG suffix optimization, prefix injection, payload smuggling via base64/ROT13, defense layers — Constitutional AI, Llama Guard, NeMo Guardrails, Azure AI Content Filter — plus battery construction and ASR reporting)

Summary

LLM red team skill domain covering offensive testing of large language model systems, generative AI applications, RAG pipelines, and agentic tool-using assistants. The skill equips the operator to discover and demonstrate semantic-layer vulnerabilities — prompt injection, jailbreaks, system-prompt extraction, training-data leakage, RAG poisoning, agentic tool abuse, and safety policy bypass — using the current open-source toolkit (promptfoo, garak, PyRIT, PurpleLlama, AI-Infra-Guard, llm-guard). Findings are mapped to the OWASP LLM Top 10 (LLM01-LLM10) and the emerging LLM-ATT&CK taxonomy, and reported with reproducible payloads, response evidence, and concrete remediation guidance.

Tools: promptfoo, garak, PyRIT, PurpleLlama (Llama-Guard / CyberSecEval), AI-Infra-Guard, llm-guard, picklescan, Burp Suite, custom Python (requests/openai), ffuf, mitmproxy

Domain: ai-red-team

Installs
6
GitHub Stars
85
First Seen
Sep 19, 2026