skill-audit-loop
Warn
Audited by Gen Agent Trust Hub on Oct 1, 2026
Risk Level: MEDIUMCOMMAND_EXECUTIONPERSISTENCEINDIRECT_PROMPT_INJECTIONDYNAMIC_EXECUTION
Full Analysis
- [COMMAND_EXECUTION]: The script
scripts/score_runs.pyexecutes shell commands sourced directly from a configuration file (suite.json) usingsubprocess.runwithshell=True. While intended for validating test results (Rule checks), this allows for arbitrary command execution based on the suite's content. - [DYNAMIC_EXECUTION]: The skill dynamically assembles and executes command-line instructions for the
claudeCLI inscripts/run_matrix.py. This includes the interpolation ofextra_argsandpromptstrings provided in the external suite configuration. - [PERSISTENCE]: The skill includes an installation script (
scripts/install_hook.py) that modifies the global Claude Codesettings.jsonfile (typically located at~/.claude/settings.json). It registers aSessionEndhook that automatically executes thescripts/capture_session.pyscript whenever a CLI session concludes, creating a persistent monitoring mechanism. - [INDIRECT_PROMPT_INJECTION]: The auditing workflow relies on analyzing existing Claude Code session transcripts (
~/.claude/projects/**/*.jsonl). Malicious instructions embedded in these historical transcripts could potentially manipulate the LLM 'judge' used during the scoring phase. - Ingestion points:
scripts/scan_field.pyandscripts/trace.pyingest raw session logs from local project directories. - Boundary markers:
scripts/trace.pyemploys basic delimiters such as[user],[assistant], and[tool]to segment the trace for the LLM judge. - Capability inventory: The skill possesses the ability to execute shell commands (
scripts/run_matrix.py), write to the file system, and modify global agent settings (scripts/install_hook.py). - Sanitization: The raw conversation text extracted from session logs is not sanitized or escaped before being interpolated into the prompt sent to the LLM judge in
scripts/score_runs.py.
Audit Metadata