The Agent Skills Directory

[COMMAND_EXECUTION]: The helper script assets/orca-eval-scan.sh contains a command injection vulnerability. It constructs a Python command string by directly interpolating a shell variable ($f) into a Python script: fpath = '$f'. If a session log file path contains a single quote or other shell-sensitive characters, it can break out of the string literal and execute arbitrary Python code. An attacker could potentially trigger this by creating a maliciously named file in a project directory that the agent is likely to scan.
[DATA_EXFILTRATION]: The skill's primary function involves reading sensitive data from ~/.claude/projects/, which contains private conversation history and session logs. While no direct network exfiltration was detected in the provided files, the exposure of these logs to the model and their processing into reports creates a significant data exposure risk if the model is later prompted to send this data externally.
[PROMPT_INJECTION]: The skill is highly vulnerable to Indirect Prompt Injection (Category 8).
Ingestion points: Reads raw JSONL logs from ~/.claude/projects/ which contain content from previous sessions, including potentially untrusted data or malicious instructions encountered during web browsing or file reading in those sessions.
Boundary markers: The skill lacks explicit boundary markers or instructions to the model to ignore any system commands or directives embedded within the ingested logs.
Capability inventory: The skill has the ability to write files to the filesystem (llmwiki/) and propose significant configuration changes (update-CLAUDE.md, add-hook, promote-to-skill).
Sanitization: The script performs minimal sanitization (newline removal and character truncation) which does not prevent instructions embedded in the text from being processed by the LLM.

orca-eval