skill-eval
Warn
Audited by Gen Agent Trust Hub on Aug 31, 2026
Risk Level: MEDIUMREMOTE_CODE_EXECUTIONCOMMAND_EXECUTIONINDIRECT_PROMPT_INJECTIONDYNAMIC_EXECUTION
Full Analysis
- [REMOTE_CODE_EXECUTION]: The skill depends on numerous Python scripts located in
scripts/skill_eval/(such asrun_eval.py,run_loop.py, andquick_validate.py) and a script at~/.claude/scripts/learning-db.pythat are not included in the provided files. Because these scripts are not available for audit, their safety and actual behavior remain unverifiable. The use of a path in the user's home directory for core logic is non-standard. - [COMMAND_EXECUTION]: The skill instructions direct the agent to execute various shell commands, including
python3module calls and theclaudeCLI, to run trigger tests, benchmarks, and optimization loops. - [INDIRECT_PROMPT_INJECTION]: The skill is designed to ingest and evaluate external, potentially adversarial content from
evals.jsonand the outputs of other skills, creating an attack surface for indirect prompt injection. - Ingestion points: The system reads
evals.jsonfor test prompts and processes execution transcripts and file outputs from the skills being benchmarked. - Boundary markers: The provided sub-agent instructions (
grader.md,comparator.md) do not include explicit delimiters or instructions to ignore embedded commands when processing these outputs. - Capability inventory: The skill has access to
Bash,Write,Read,Grep,Glob, andAgenttools, which could be abused if the agent is influenced by malicious instructions in the data it evaluates. - Sanitization: There is no description of sanitization, filtering, or validation of the input prompts or the skill outputs before they are processed by the evaluation logic.
- [DYNAMIC_EXECUTION]: The skill implements automated self-improvement and optimization processes (
run_loop.py,self-improve-loop.md) that generate new variants ofSKILL.mdinstruction files and promote them to replace the original definitions. The automated generation and promotion of executable instructions constitutes dynamic execution of generated content.
Audit Metadata