devtu-benchmark-harness
Warn
Audited by Snyk on May 29, 2026
Risk Level: MEDIUM
Full Analysis
MEDIUM W011: Third-party content exposure detected (indirect prompt injection risk).
- Third-party content exposure detected (high risk: 0.85). At runtime,
scripts/run_eval.pyreads BixBench capsule files (e.g.,capsule_pathcontents) and injects their readable text into the LLM prompt viaprepare_prompt()andprecompute_for_capsule()(e.g., CSV/metadata-derived outputs and “Data files are located at…” + file listings), so the LLM context can include outsider-authored free text from the capsule dataset.
Issues (1)
W011
MEDIUMThird-party content exposure detected (indirect prompt injection risk).
Audit Metadata