model-compare
Fail
Audited by Gen Agent Trust Hub on Aug 30, 2026
Risk Level: HIGHDYNAMIC_EXECUTIONCOMMAND_EXECUTIONINDIRECT_PROMPT_INJECTIONEXTERNAL_DOWNLOADS
Full Analysis
- [DYNAMIC_EXECUTION]: The
run_pythontool inscripts/compare.pyallows for the execution of arbitrary Python code generated by the AI models during comparisons. This code is written to a temporary file and executed via the system's Python interpreter. While the script implements a 10-second timeout and uses a temporary directory, it does not provide true sandboxing or network isolation for the executed code. - [COMMAND_EXECUTION]: The script uses
subprocess.runto execute Python code for therun_pythontool and to run theddgsCLI for web search fallbacks. Additionally,SKILL.mdprovides an example command that pipes output fromcurldirectly into the Python interpreter. - [INDIRECT_PROMPT_INJECTION]: The skill is vulnerable to indirect prompt injection because it ingests untrusted content from the web via the
web_extracttool and provides it to models that have access to powerful tools likerun_pythonandwrite_file. - Ingestion points: Untrusted data enters the context through the
execute_web_extractfunction inscripts/compare.py, which fetches content from arbitrary URLs provided by the model. - Boundary markers: The ingested content is not wrapped in clear delimiters or accompanied by instructions to ignore embedded commands before being processed by the model.
- Capability inventory: The skill possesses high-tier capabilities, including arbitrary code execution (
run_python), filesystem modification (write_file), and filesystem access (read_file). - Sanitization: The
execute_web_extracttool performs basic HTML tag stripping but does not sanitize the text for potential prompt injection payloads. - [EXTERNAL_DOWNLOADS]: The skill makes network requests to several well-known AI service providers, including OpenRouter, NVIDIA, and Ollama Cloud, to fetch model lists and chat completions. It also performs downloads from arbitrary user- or model-supplied URLs through its web research functionality.
Recommendations
- HIGH: Downloads and executes remote code from: https://openrouter.ai/api/v1/models, unknown (check file) - DO NOT USE without thorough review
Audit Metadata