rag-eval
Fail
Audited by Gen Agent Trust Hub on May 9, 2026
Risk Level: HIGHDATA_EXFILTRATIONCOMMAND_EXECUTIONPROMPT_INJECTION
Full Analysis
- [DATA_EXFILTRATION]: The
scripts/session_ingest.pyscript accesses sensitive local directories including~/.claude/projects/and~/.skill-studio/sessions/. These directories contain transcripts and session history from other AI tools, which may expose sensitive information from unrelated projects or conversations. - [COMMAND_EXECUTION]: The skill instructions direct the agent to execute local Python scripts (
scripts/session_ingest.pyandscripts/eval_sweep.py). The inclusion of user-provided variables like<session_id>in shell commands creates a risk of command injection if the input is not strictly sanitized by the execution environment. - [PROMPT_INJECTION]: The skill has a significant indirect prompt injection surface as it ingests untrusted data from the user's dataset, gold-set, and session transcripts.
- Ingestion points: Data enters the agent context via the dataset corpus, the Q&A gold-set, and session transcripts processed by
scripts/session_ingest.py. - Boundary markers: The instructions do not specify any delimiters or safety warnings to ignore instructions embedded within the ingested data.
- Capability inventory: The skill executes local Python scripts (
subprocessequivalent) and has network capabilities viatavily-searchandfirecrawl-research. - Sanitization: There is no evidence of sanitization or filtering of the external content before it is used to plan or run evaluation sweeps.
Recommendations
- AI detected serious security threats
Audit Metadata