devtu-benchmark-harness

Warn

Audited by Socket on Oct 7, 2026

2 alerts found:

Anomalyx2
AnomalyLOW
scripts/run_harness_loop.sh

The script appears to be a legitimate benchmark orchestration utility and contains no direct evidence of malware or data exfiltration. It has a meaningful security weakness: user-controlled --seed and --retest values are interpolated into Python source, enabling arbitrary Python code execution when crafted values are supplied. --benchmark and related paths also lack visible validation, creating path-selection risks. Inputs should be passed as positional arguments or environment-independent data, with integer/type validation and canonical path or allowlist checks.

Confidence: 98%Severity: 67%
AnomalyLOW
scripts/run_one_by_one.sh

The code is a legitimate benchmark harness with no clear malware indicators. It has a moderate security weakness because command-line paths and question_id values are inserted into dynamically constructed Python commands without safe argument passing or escaping. An attacker able to control those values could potentially achieve Python code execution under the invoking user's account. The risk is lower when inputs and repository contents are trusted.

Confidence: 97%Severity: 57%
Audit Metadata
Analyzed At
Oct 7, 2026, 01:49 PM
Package URL
pkg:socket/skills-sh/mims-harvard%2Ftooluniverse%2Fdevtu-benchmark-harness%2F@ea3b930323e270c81682795091bacc44d87e907445d368a85f54b01a9a4851c9