eval-harness-first
Pass
Audited by Gen Agent Trust Hub on Jul 14, 2026
Risk Level: SAFECOMMAND_EXECUTIONREMOTE_CODE_EXECUTIONPROMPT_INJECTION
Full Analysis
- [REMOTE_CODE_EXECUTION]: The references/grader-templates.md file contains a template for an execution-based grader that writes model completions to files and executes them. While the documentation mandates the use of a secure sandbox, this pattern involves running untrusted code generated by an LLM.
- [COMMAND_EXECUTION]: The skill uses subprocess.run within the execution grader template in references/grader-templates.md to call pytest, providing a mechanism for command execution on the host where the harness is deployed.
- [PROMPT_INJECTION]: The skill is designed to process external agent traces for evaluation, which introduces a surface for indirect prompt injection. 1. Ingestion points: Production and agent traces are gathered as defined in SKILL.md. 2. Boundary markers: The references/grader-templates.md file defines isolation requirements and sandboxing via sandbox_cmd. 3. Capability inventory: The harness includes capabilities for code execution via subprocess.run and LLM-based judging in references/grader-templates.md. 4. Sanitization: The skill relies on isolation rather than input validation or sanitization as noted in references/grader-templates.md.
Audit Metadata