orchestrating-llm-attacks-with-pyrit
Fail
Audited by Snyk on Jun 24, 2026
Risk Level: CRITICAL
Full Analysis
CRITICAL E006: Malicious code pattern detected in skill scripts.
- Malicious code pattern detected (high risk: 1.00). This repository intentionally implements automated multi-turn adversarial LLM attacks (prompt injection / jailbreak), includes obfuscation converters to evade filters, and can route and persist target conversation content to external adversarial/scoring endpoints or exports — enabling data exfiltration and misuse.
MEDIUM W011: Third-party content exposure detected (indirect prompt injection risk).
- Third-party content exposure detected (high risk: 0.85). The required workflow runs multi-turn attacks where the LLM context is populated with the adversarial chat model’s generated messages and the target model’s responses (both are outsider-authored text from models the user did not author), via
orchestrator.run_attack_async(...)and subsequentresult.print_conversation_async()/memory export.
MEDIUM W012: Unverifiable external dependency detected (runtime URL that controls agent).
- Potentially malicious external URL detected (high risk: 0.90). The skill instantiates OpenAIChatTarget (example endpoint "https://api.openai.com/v1/chat/completions") at runtime for both the adversarial and scorer models, and the responses fetched from that API are injected as prompts that drive the target model, so remote content directly controls agent instructions.
Issues (3)
E006
CRITICALMalicious code pattern detected in skill scripts.
W011
MEDIUMThird-party content exposure detected (indirect prompt injection risk).
W012
MEDIUMUnverifiable external dependency detected (runtime URL that controls agent).
Audit Metadata