orca-eval
Pass
Audited by Gen Agent Trust Hub on Sep 20, 2026
Risk Level: SAFECOMMAND_EXECUTIONDATA_EXFILTRATIONINDIRECT_PROMPT_INJECTION
Full Analysis
- [COMMAND_EXECUTION]: The skill executes a local shell script
assets/orca-eval-scan.shto parse session logs. While this script is provided within the skill assets, executing shell scripts with user-supplied arguments (the number of sessionsN) is a capability that should be monitored. - [DATA_EXPOSURE]: The skill reads session log files from
~/.claude/projects/. These files contain the history of interactions, which may include sensitive code, prompts, and tool outputs. This access is necessary for the skill's primary purpose of session distillation but involves handling private data. - [INDIRECT_PROMPT_INJECTION]: The skill processes historical session data to identify 'Best Practices' and 'Actions'.
- Ingestion points: Reads
*.jsonlfiles from the user's local Claude session directory viaassets/orca-eval-scan.sh. - Boundary markers: The skill uses markdown tables and structured report templates to separate findings, but does not implement strict delimiters to prevent the agent from following instructions embedded within the logs being analyzed.
- Capability inventory: The skill can write reports to the filesystem (
llmwiki/wiki/draft/orca/) and update index files. It also triggers further agent actions through a 'proposal' mechanism. - Sanitization: The underlying Python script truncates extracted strings (prompts to 200 chars, errors to 150 chars) and sanitizes newline characters, which provides basic mitigation against simple payload execution.
- [DYNAMIC_EXECUTION]: The
assets/orca-eval-scan.shscript executes an embedded Python script usingpython3 -c. The Python code is hardcoded in the asset file and used to process the JSONL log format.
Audit Metadata