orca-replay

Warn

Audited by Gen Agent Trust Hub on Oct 27, 2026

Risk Level: MEDIUMCOMMAND_EXECUTIONDYNAMIC_EXECUTIONDATA_EXFILTRATIONINDIRECT_PROMPT_INJECTION
Full Analysis
  • [COMMAND_EXECUTION]: The orca_replay tool re-executes shell commands captured in a recording. This means any command issued during the original session—including those that reach the network, modify databases, or interact with local infrastructure—will be run again on the host machine.
  • Evidence: The 'Workflow' section explicitly states that re-executing the agent involves running every shell command it issued again for real, including those that reached outside the working tree.
  • [DYNAMIC_EXECUTION]: The skill uses dynamic execution for its primary functionality and for verifying comparison results.
  • Evidence: The orca_compare tool accepts a verify argument which is a shell command (e.g., npm test) used to determine the success of a fork. The skill warns against using npx to prevent the download and execution of unreviewed packages.
  • [DATA_EXFILTRATION]: The orca_compare tool transmits recorded trace data to external model providers to fork and compare agent runs.
  • Evidence: The documentation notes that orca_compare uploads the recording (which may include source code, prompts, and credentials) to multiple providers. While it recommends using orca scrub, it admits automated scrubbing is not guaranteed.
  • [INDIRECT_PROMPT_INJECTION]: The skill is designed to ingest and process recorded agent runs which are explicitly described as "untrusted evidence."
  • Ingestion points: Recorded traces stored under the .orca/runs directory.
  • Boundary markers: The skill includes strong instructions to the agent to treat trace content as inert data and never as instructions, though this is a manual instruction rather than a technical enforcement.
  • Capability inventory: The skill can execute shell commands (orca_replay), perform network operations (orca_compare), and read/write to the filesystem.
  • Sanitization: No automatic sanitization of trace content is mentioned, though manual scrubbing is suggested for exports.
Audit Metadata
Risk Level
MEDIUM
Analyzed
Oct 27, 2026, 04:01 AM