create-agent-tests
Fail
Audited by Gen Agent Trust Hub on Jun 29, 2026
Risk Level: HIGHCOMMAND_EXECUTIONREMOTE_CODE_EXECUTIONPROMPT_INJECTION
Full Analysis
- [COMMAND_EXECUTION]: The orchestration script
assets/run-agent-verification.tsfacilitates arbitrary shell command execution through the--engine-command-templateflag. The implementation usesspawnSyncwithshell: trueand performs simple string replacement (e.g.,{{prompt_file}},{{case_path}}) without proper shell escaping or sanitization, which can lead to command injection if file paths or arguments contain malicious shell characters. - [REMOTE_CODE_EXECUTION]: The runner script explicitly includes the
--permission-mode bypassPermissionsflag when invoking theclaudeengine. This setting removes the standard 'human-in-the-loop' safety mechanism, allowing the agent to perform high-risk actions—such as file modification, network communication, or terminal command execution—without user oversight or confirmation. - [COMMAND_EXECUTION]: The script allows the engine binary to be specified via the
AGENT_VERIFICATION_ENGINEenvironment variable or the--engineflag. This provided binary is then executed viaspawnSync, creating a risk that an attacker could trick the runner into executing unauthorized or malicious binaries if they can influence the environment configuration. - [PROMPT_INJECTION]: The skill is vulnerable to Indirect Prompt Injection because it ingests external Markdown test files and includes their content directly in the agent's instructions with high-level capabilities.
- Ingestion points: Instructions are loaded from files at
tests/agent/<feature>/<case>.mdand read into thecaseMarkdownvariable inassets/run-agent-verification.ts. - Boundary markers: The prompt uses the directive "Execute the case below exactly" without any delimiters, escaping, or "ignore embedded instructions" warnings for the ingested content.
- Capability inventory: The executing agent is granted extensive system access, including browser automation, terminal command execution, database queries, and cloud provider API access.
- Sanitization: No validation, filtering, or escaping is performed on the Markdown content, allowing any instructions embedded in the test files to be executed with the agent's full privileges.
Recommendations
- AI detected serious security threats
Audit Metadata