model-compare

Fail

Audited by Gen Agent Trust Hub on Oct 5, 2026

Risk Level: HIGHREMOTE_CODE_EXECUTIONDYNAMIC_EXECUTIONINDIRECT_PROMPT_INJECTIONCOMMAND_EXECUTION
Full Analysis
  • [DYNAMIC_EXECUTION]: The run_python tool in scripts/compare.py allows models to generate and execute arbitrary Python code using subprocess.run. Although it uses a temporary directory and a 10-second timeout to limit local impact, the implementation fails to provide the network isolation described in the tool's documentation. This allows models to execute code that can perform unauthorized network requests or access system information.
  • [REMOTE_CODE_EXECUTION]: The skill facilitates the execution of code sourced from remote, untrusted LLM responses. An attacker could influence a model to return a malicious payload that the skill then executes on the host system with the privileges of the agent.
  • [INDIRECT_PROMPT_INJECTION]: The web_extract and web_search tools in scripts/compare.py ingest content from arbitrary external URLs. This data is fed back into the model's context without sufficient sanitization or instruction-guarding wrappers, creating a surface where malicious web content could override the agent's instructions.
  • Ingestion points: execute_web_search and execute_web_extract functions in scripts/compare.py.
  • Boundary markers: Absent; tool results are passed directly into the conversation history without protective delimiters.
  • Capability inventory: run_python (code execution), write_file (filesystem writes), and web_search (network operations).
  • Sanitization: execute_web_extract performs basic HTML tag stripping and truncates output to 3000 characters, which does not prevent instructional content from being processed.
  • [COMMAND_EXECUTION]: The scripts/compare.py script makes frequent use of subprocess.run to execute dynamic scripts and the ddgs CLI. Additionally, SKILL.md includes examples of piping remote API responses from openrouter.ai (a well-known service) directly into a Python interpreter command string.
Recommendations
  • HIGH: Downloads and executes remote code from: https://openrouter.ai/api/v1/models, unknown (check file) - DO NOT USE without thorough review
  • AI detected serious security threats
Audit Metadata
Risk Level
HIGH
Analyzed
Oct 5, 2026, 09:55 PM