devtu-benchmark-harness

Warn

Audited by Snyk on May 29, 2026

Risk Level: MEDIUM
Full Analysis

MEDIUM W011: Third-party content exposure detected (indirect prompt injection risk).

  • Third-party content exposure detected (high risk: 0.85). At runtime, scripts/run_eval.py reads BixBench capsule files (e.g., capsule_path contents) and injects their readable text into the LLM prompt via prepare_prompt() and precompute_for_capsule() (e.g., CSV/metadata-derived outputs and “Data files are located at…” + file listings), so the LLM context can include outsider-authored free text from the capsule dataset.

Issues (1)

W011
MEDIUM

Third-party content exposure detected (indirect prompt injection risk).

Audit Metadata
Risk Level
MEDIUM
Analyzed
May 29, 2026, 05:25 AM
Issues
1
Security Audit — snyk — devtu-benchmark-harness