The Agent Skills Directory

[COMMAND_EXECUTION]: The script scripts/runner.py executes shell commands defined in scenario.setup_commands using subprocess.run. These commands are dynamically generated by an LLM based on the content of the skill file being analyzed, which could lead to arbitrary command execution if a malicious input file manipulates the generation process.
[COMMAND_EXECUTION]: Multiple scripts including scripts/classifier.py, scripts/runner.py, scripts/scenario_generator.py, and scripts/spec_generator.py use subprocess.run to invoke the claude CLI tool for processing prompts and running scenarios.
[PROMPT_INJECTION]: The tool is vulnerable to indirect prompt injection (Category 8). It ingests untrusted data (skill/rule files) at scripts/run.py, generates prompts from this data in scripts/scenario_generator.py, and executes them in scripts/runner.py. A malicious skill file could contain instructions designed to hijack the behavior of the evaluation agent during the scenario run. Although the runner uses sandbox directories and limits allowed tools to Read, Write, Edit, Bash, Glob, Grep, these capabilities are still significant.
[COMMAND_EXECUTION]: scripts/runner.py executes git init and other setup commands within a temporary sandbox directory located at /tmp/skill-comply-sandbox. While path traversal protections are implemented, the execution of generated setup commands remains a risk factor.

skill-comply