octocode-graph-eval
Pass
Audited by Gen Agent Trust Hub on Aug 26, 2026
Risk Level: SAFEPROMPT_INJECTIONCOMMAND_EXECUTION
Full Analysis
- [INDIRECT_PROMPT_INJECTION]: The skill is designed to ingest and process external data such as execution traces and datasets for failure analysis and grading.
- Ingestion points:
references/error-analysis.mdinstructs the agent to gather representative traces from production or synthetic sources.references/trajectory-grading.mdinvolves evaluating tool-call sequences from various runs. - Boundary markers: The instructions for using 'LLM judges' in
references/eval-techniques.mddo not specify the use of delimiters or instructions for the judge to ignore malicious content embedded within the traces. - Capability inventory: The skill possesses the capability to execute local scripts (
scripts/*.mjs) and perform autonomous file mutations during 'keep/discard' loops as defined inreferences/agent-loop.md. - Sanitization: No sanitization or escaping mechanisms are prescribed for the external trace content before it is processed by the agent or submitted to LLM-based grading rubrics.
- [PROMPT_INJECTION]: Instructions within the skill promote a high degree of autonomy that explicitly avoids human oversight during execution loops.
- Evidence:
references/agent-loop.mdcontains the instruction: "Do not pause mid-loop for permission unless the user interrupted." Similarly,references/karpathy-patterns.mdcites the pattern: "NEVER STOP until interrupted." - These instructions increase the risk that an agent redirected by a prompt injection (direct or indirect) would continue executing malicious actions without a verification gate.
- [COMMAND_EXECUTION]: The skill includes and relies upon the execution of several JavaScript files to function correctly.
- Evidence:
scripts/loop-report.mjs,scripts/eval-eval.mjs, andscripts/check-description.mjsare provided within the skill folder. The agent is instructed to run these using Node.js for self-testing and reporting tasks. These scripts use the standardnode:fsmodule to read and validate local files.
Audit Metadata