model-compare

Fail

Audited by Gen Agent Trust Hub on Aug 30, 2026

Risk Level: HIGHDYNAMIC_EXECUTIONCOMMAND_EXECUTIONINDIRECT_PROMPT_INJECTIONEXTERNAL_DOWNLOADS
Full Analysis
  • [DYNAMIC_EXECUTION]: The run_python tool in scripts/compare.py allows for the execution of arbitrary Python code generated by the AI models during comparisons. This code is written to a temporary file and executed via the system's Python interpreter. While the script implements a 10-second timeout and uses a temporary directory, it does not provide true sandboxing or network isolation for the executed code.
  • [COMMAND_EXECUTION]: The script uses subprocess.run to execute Python code for the run_python tool and to run the ddgs CLI for web search fallbacks. Additionally, SKILL.md provides an example command that pipes output from curl directly into the Python interpreter.
  • [INDIRECT_PROMPT_INJECTION]: The skill is vulnerable to indirect prompt injection because it ingests untrusted content from the web via the web_extract tool and provides it to models that have access to powerful tools like run_python and write_file.
  • Ingestion points: Untrusted data enters the context through the execute_web_extract function in scripts/compare.py, which fetches content from arbitrary URLs provided by the model.
  • Boundary markers: The ingested content is not wrapped in clear delimiters or accompanied by instructions to ignore embedded commands before being processed by the model.
  • Capability inventory: The skill possesses high-tier capabilities, including arbitrary code execution (run_python), filesystem modification (write_file), and filesystem access (read_file).
  • Sanitization: The execute_web_extract tool performs basic HTML tag stripping but does not sanitize the text for potential prompt injection payloads.
  • [EXTERNAL_DOWNLOADS]: The skill makes network requests to several well-known AI service providers, including OpenRouter, NVIDIA, and Ollama Cloud, to fetch model lists and chat completions. It also performs downloads from arbitrary user- or model-supplied URLs through its web research functionality.
Recommendations
  • HIGH: Downloads and executes remote code from: https://openrouter.ai/api/v1/models, unknown (check file) - DO NOT USE without thorough review
Audit Metadata
Risk Level
HIGH
Analyzed
Aug 30, 2026, 08:30 PM
Security Audit — agent-trust-hub — model-compare