meta-harness-terminal-bench-2
Warn
Audited by Gen Agent Trust Hub on Jul 5, 2026
Risk Level: MEDIUMREMOTE_CODE_EXECUTIONCOMMAND_EXECUTIONPROMPT_INJECTION
Full Analysis
- [REMOTE_CODE_EXECUTION]: The skill instructions explicitly direct the agent to generate arbitrary Python code in the
agents/directory. This includes overriding methods, calling libraries, and making raw API calls. The skill then executes this generated code during the 'Smoke test' phase by importing it. - Evidence: 'The search space is arbitrary Python code. You can override any method, call any library, make raw API calls...'
- Evidence: 'Smoke test: validate import (uv run python -c "from agents. import *; print('OK')")'
- [COMMAND_EXECUTION]: The agent is instructed to use shell commands (
uv run) to execute Python code it has created. This bypasses typical safety constraints by executing model-generated logic in a sub-process. - [PROMPT_INJECTION]: The skill is vulnerable to Indirect Prompt Injection (Category 8) because it ingests untrusted data from previous agent trajectories and uses it to synthesize new executable code.
- Ingestion points: The agent reads state files (
frontier_val.json,evolution_summary.jsonl) and 'deep-reads failed AND successful trajectories' fromjobs/andlogs/directories. - Boundary markers: There are no instructions provided to treat the log content as untrusted or to ignore embedded instructions within those logs.
- Capability inventory: The agent has the capability to write arbitrary Python files and execute shell commands via sub-processes.
- Sanitization: No sanitization or validation logic is specified for the data read from external logs before it is used to inform the generation of the next agent variant.
Audit Metadata