devtu-benchmark-harness
Pass
Audited by Gen Agent Trust Hub on May 29, 2026
Risk Level: SAFE
Full Analysis
- [COMMAND_EXECUTION]: The skill's scripts, including run_eval.py, grade_answers.py, and solve_one.sh, use subprocess calls to invoke the Claude CLI and other local bioinformatics tools like phykit. This is the intended behavior for an orchestrated benchmarking harness designed to automate testing and evaluation.
- [EXTERNAL_DOWNLOADS]: The documentation references instructions for the user to install R packages and download benchmark data capsules using provided scripts. These are standard setup procedures for bioinformatics workflows.
- [INDIRECT_PROMPT_INJECTION]: The grading system and evaluation loop ingest data from external benchmark files (questions, ground truths, and tool outputs) and include them in prompts for the agent. The skill uses delimiters and instructional framing to manage the risk of adversarial content within the datasets, which is appropriate for its use in controlled testing environments.
- [DYNAMIC_EXECUTION]: The harness includes a pre-execution feature that automatically runs specific local scripts based on the file patterns found in data capsules to provide deterministic results to the agent. This feature utilizes local, repository-contained scripts and is used to improve benchmark accuracy.
Audit Metadata