run-evals
Warn
Audited by Socket on Sep 15, 2026
1 alert found:
SecuritySecurityreferences/end-to-end-examples.md
MEDIUMSecurityMEDIUM
references/end-to-end-examples.md
The code is primarily documentation for model-evaluation workflows and contains no clear supply-chain malware indicators. However, Example 3 has a significant code-execution risk: untrusted model-generated or dataset-provided Python is passed directly to exec and eval without sandboxing. This could result in arbitrary code execution if the benchmark, model output, or test expression is compromised. External API transmission is expected but may create data-privacy considerations. Review or remove the exec/eval design and use an isolated sandbox or a restricted test protocol.
Confidence: 98%Severity: 72%
Audit Metadata