shiplight

Pass

Audited by Gen Agent Trust Hub on Oct 7, 2026

Risk Level: SAFECOMMAND_EXECUTIONDYNAMIC_EXECUTIONPRIVILEGE_ESCALATIONINDIRECT_PROMPT_INJECTIONDATA_EXFILTRATIONEXTERNAL_DOWNLOADS
Full Analysis
  • [COMMAND_EXECUTION]: The bundled run-agent-verification.ts script uses child_process.spawn and spawnSync to execute verification engines (codex, claude) and customizable shell templates. It also performs automated package installations via npm install and npm ci within project subdirectories to set up testing harnesses.
  • [DYNAMIC_EXECUTION]: The Shiplight YAML test format supports a js: key that executes arbitrary JavaScript within the Node.js/Playwright context. This is used for assertions, loop conditions, and wait predicates. While essential for complex testing logic, it represents a dynamic execution surface for code provided in test specifications.
  • [PRIVILEGE_ESCALATION]: When running agent-based verifications, the toolkit executes the codex engine with the --sandbox danger-full-access flag. This configuration intentionally grants the sub-agent elevated permissions to interact with databases, network resources, and MCP tools necessary for full-stack testing.
  • [INDIRECT_PROMPT_INJECTION]: The skill possesses a broad attack surface for indirect prompt injection as it ingests and processes untrusted project files, Markdown specifications, and live browser data to generate and execute tests.
  • Ingestion points: Processes specs/tests/*.md, TESTING.md, source code, browser DOM/screenshots, and llms.txt files.
  • Boundary markers: Utilizes prompt templates in run-agent-verification.ts to separate instructions from the data being analyzed.
  • Capability inventory: Access to shell execution, filesystem writes, and browser automation via MCP tools.
  • Sanitization: Implements pattern-based redaction to scrub secrets from diagnostic reports and encourages the use of environment variables for credential management.
  • [DATA_EXFILTRATION]: The skill manages sensitive data, including authentication cookies and storage states in ~/.shiplight/ and API tokens in .env. It communicates with the vendor's cloud API (api.shiplight.ai) to upload test results and diagnostic support tickets.
  • [EXTERNAL_DOWNLOADS]: The skill performs self-updates and dependency management by fetching the shiplightai and skills packages from the NPM registry and using the ShiplightAI/ci-triage GitHub Action. These resources originate from the toolkit's official vendor.
Audit Metadata
Risk Level
SAFE
Analyzed
Oct 7, 2026, 02:40 PM