create-agent-tests

Fail

Audited by Gen Agent Trust Hub on Jun 29, 2026

Risk Level: HIGHCOMMAND_EXECUTIONREMOTE_CODE_EXECUTIONPROMPT_INJECTION
Full Analysis
  • [COMMAND_EXECUTION]: The orchestration script assets/run-agent-verification.ts facilitates arbitrary shell command execution through the --engine-command-template flag. The implementation uses spawnSync with shell: true and performs simple string replacement (e.g., {{prompt_file}}, {{case_path}}) without proper shell escaping or sanitization, which can lead to command injection if file paths or arguments contain malicious shell characters.
  • [REMOTE_CODE_EXECUTION]: The runner script explicitly includes the --permission-mode bypassPermissions flag when invoking the claude engine. This setting removes the standard 'human-in-the-loop' safety mechanism, allowing the agent to perform high-risk actions—such as file modification, network communication, or terminal command execution—without user oversight or confirmation.
  • [COMMAND_EXECUTION]: The script allows the engine binary to be specified via the AGENT_VERIFICATION_ENGINE environment variable or the --engine flag. This provided binary is then executed via spawnSync, creating a risk that an attacker could trick the runner into executing unauthorized or malicious binaries if they can influence the environment configuration.
  • [PROMPT_INJECTION]: The skill is vulnerable to Indirect Prompt Injection because it ingests external Markdown test files and includes their content directly in the agent's instructions with high-level capabilities.
  • Ingestion points: Instructions are loaded from files at tests/agent/<feature>/<case>.md and read into the caseMarkdown variable in assets/run-agent-verification.ts.
  • Boundary markers: The prompt uses the directive "Execute the case below exactly" without any delimiters, escaping, or "ignore embedded instructions" warnings for the ingested content.
  • Capability inventory: The executing agent is granted extensive system access, including browser automation, terminal command execution, database queries, and cloud provider API access.
  • Sanitization: No validation, filtering, or escaping is performed on the Markdown content, allowing any instructions embedded in the test files to be executed with the agent's full privileges.
Recommendations
  • AI detected serious security threats
Audit Metadata
Risk Level
HIGH
Analyzed
Jun 29, 2026, 05:16 PM
Security Audit — agent-trust-hub — create-agent-tests