qa-generating-e2e-tests
Fail
Audited by Gen Agent Trust Hub on Apr 20, 2026
Risk Level: CRITICALREMOTE_CODE_EXECUTIONCOMMAND_EXECUTIONPROMPT_INJECTION
Full Analysis
- [REMOTE_CODE_EXECUTION]: The script
scripts/generate_playwright.pycreates executable Python code by directly interpolating journey definition data into string templates without sanitization.\n - Evidence: In
generate_playwright.py, thegenerate_step_codeandgenerate_test_filefunctions use f-strings to insert variables likevalue,target, andbase_urlinto the generated Python script (e.g.,lines.append(f' await {locator}.fill("{value}")')).\n - Risk: Maliciously crafted input in the
journeys.jsonfile (which could be derived from untrusted requirements or application discovery) can escape the string literals and inject arbitrary Python code into the generated test files.\n- [COMMAND_EXECUTION]: The skill facilitates the execution of the generated Python scripts through therun_e2e.pyscript as described in the workflow.\n - Evidence: The
SKILL.mdworkflow encourages running the generated tests immediately after generation (python scripts/run_e2e.py --tests tests/ ...).\n - Risk: Since the generation phase is vulnerable to code injection, the subsequent execution phase provides a direct path for an attacker to achieve command execution on the host system.\n- [PROMPT_INJECTION]: The skill processes potentially untrusted data from external sources to define user journeys, creating an attack surface for indirect injection.\n
- Ingestion points: Data enters via
journey_from_requirements.py(which parses Jira tickets or requirements files) anddiscover_journeys.py(which crawls a target URL).\n - Boundary markers: None identified in the provided code; input is treated as trusted data during the code generation process.\n
- Capability inventory: The skill has the ability to write files (
generate_playwright.pyusingPath.write_text) and execute shell commands as shown in the workflow.\n - Sanitization: There is no evidence of input validation, escaping, or sanitization of the data retrieved from external sources before it is used in code generation.
Recommendations
- AI detected serious security threats
Audit Metadata