skills/rheinmir/setup/orca-eval/Gen Agent Trust Hub

orca-eval

Pass

Audited by Gen Agent Trust Hub on Sep 20, 2026

Risk Level: SAFECOMMAND_EXECUTIONDATA_EXFILTRATIONINDIRECT_PROMPT_INJECTION
Full Analysis
  • [COMMAND_EXECUTION]: The skill executes a local shell script assets/orca-eval-scan.sh to parse session logs. While this script is provided within the skill assets, executing shell scripts with user-supplied arguments (the number of sessions N) is a capability that should be monitored.
  • [DATA_EXPOSURE]: The skill reads session log files from ~/.claude/projects/. These files contain the history of interactions, which may include sensitive code, prompts, and tool outputs. This access is necessary for the skill's primary purpose of session distillation but involves handling private data.
  • [INDIRECT_PROMPT_INJECTION]: The skill processes historical session data to identify 'Best Practices' and 'Actions'.
  • Ingestion points: Reads *.jsonl files from the user's local Claude session directory via assets/orca-eval-scan.sh.
  • Boundary markers: The skill uses markdown tables and structured report templates to separate findings, but does not implement strict delimiters to prevent the agent from following instructions embedded within the logs being analyzed.
  • Capability inventory: The skill can write reports to the filesystem (llmwiki/wiki/draft/orca/) and update index files. It also triggers further agent actions through a 'proposal' mechanism.
  • Sanitization: The underlying Python script truncates extracted strings (prompts to 200 chars, errors to 150 chars) and sanitizes newline characters, which provides basic mitigation against simple payload execution.
  • [DYNAMIC_EXECUTION]: The assets/orca-eval-scan.sh script executes an embedded Python script using python3 -c. The Python code is hardcoded in the asset file and used to process the JSONL log format.
Audit Metadata
Risk Level
SAFE
Analyzed
Sep 20, 2026, 03:25 AM