skill-audit-loop

Warn

Audited by Gen Agent Trust Hub on Oct 1, 2026

Risk Level: MEDIUMCOMMAND_EXECUTIONPERSISTENCEINDIRECT_PROMPT_INJECTIONDYNAMIC_EXECUTION
Full Analysis
  • [COMMAND_EXECUTION]: The script scripts/score_runs.py executes shell commands sourced directly from a configuration file (suite.json) using subprocess.run with shell=True. While intended for validating test results (Rule checks), this allows for arbitrary command execution based on the suite's content.
  • [DYNAMIC_EXECUTION]: The skill dynamically assembles and executes command-line instructions for the claude CLI in scripts/run_matrix.py. This includes the interpolation of extra_args and prompt strings provided in the external suite configuration.
  • [PERSISTENCE]: The skill includes an installation script (scripts/install_hook.py) that modifies the global Claude Code settings.json file (typically located at ~/.claude/settings.json). It registers a SessionEnd hook that automatically executes the scripts/capture_session.py script whenever a CLI session concludes, creating a persistent monitoring mechanism.
  • [INDIRECT_PROMPT_INJECTION]: The auditing workflow relies on analyzing existing Claude Code session transcripts (~/.claude/projects/**/*.jsonl). Malicious instructions embedded in these historical transcripts could potentially manipulate the LLM 'judge' used during the scoring phase.
  • Ingestion points: scripts/scan_field.py and scripts/trace.py ingest raw session logs from local project directories.
  • Boundary markers: scripts/trace.py employs basic delimiters such as [user], [assistant], and [tool] to segment the trace for the LLM judge.
  • Capability inventory: The skill possesses the ability to execute shell commands (scripts/run_matrix.py), write to the file system, and modify global agent settings (scripts/install_hook.py).
  • Sanitization: The raw conversation text extracted from session logs is not sanitized or escaped before being interpolated into the prompt sent to the LLM judge in scripts/score_runs.py.
Audit Metadata
Risk Level
MEDIUM
Analyzed
Oct 1, 2026, 04:18 PM