octocode-graph-eval

Pass

Audited by Gen Agent Trust Hub on Aug 26, 2026

Risk Level: SAFEPROMPT_INJECTIONCOMMAND_EXECUTION
Full Analysis
  • [INDIRECT_PROMPT_INJECTION]: The skill is designed to ingest and process external data such as execution traces and datasets for failure analysis and grading.
  • Ingestion points: references/error-analysis.md instructs the agent to gather representative traces from production or synthetic sources. references/trajectory-grading.md involves evaluating tool-call sequences from various runs.
  • Boundary markers: The instructions for using 'LLM judges' in references/eval-techniques.md do not specify the use of delimiters or instructions for the judge to ignore malicious content embedded within the traces.
  • Capability inventory: The skill possesses the capability to execute local scripts (scripts/*.mjs) and perform autonomous file mutations during 'keep/discard' loops as defined in references/agent-loop.md.
  • Sanitization: No sanitization or escaping mechanisms are prescribed for the external trace content before it is processed by the agent or submitted to LLM-based grading rubrics.
  • [PROMPT_INJECTION]: Instructions within the skill promote a high degree of autonomy that explicitly avoids human oversight during execution loops.
  • Evidence: references/agent-loop.md contains the instruction: "Do not pause mid-loop for permission unless the user interrupted." Similarly, references/karpathy-patterns.md cites the pattern: "NEVER STOP until interrupted."
  • These instructions increase the risk that an agent redirected by a prompt injection (direct or indirect) would continue executing malicious actions without a verification gate.
  • [COMMAND_EXECUTION]: The skill includes and relies upon the execution of several JavaScript files to function correctly.
  • Evidence: scripts/loop-report.mjs, scripts/eval-eval.mjs, and scripts/check-description.mjs are provided within the skill folder. The agent is instructed to run these using Node.js for self-testing and reporting tasks. These scripts use the standard node:fs module to read and validate local files.
Audit Metadata
Risk Level
SAFE
Analyzed
Aug 26, 2026, 05:42 AM
Security Audit — agent-trust-hub — octocode-graph-eval