swe-bench
Pass
Audited by Gen Agent Trust Hub on Aug 1, 2026
Risk Level: SAFE
Full Analysis
- [COMMAND_EXECUTION]: The skill executes command-line operations using the
opensquillaCLI tool (e.g.,opensquilla swebench solve,opensquilla swebench pull). These commands are part of the vendor's own software suite and are used to manage the benchmark lifecycle. - [EXTERNAL_DOWNLOADS]: The skill installs the
opensquilla[swebench]package viauvorpip. As this package belongs to the skill's author (OpenSquilla), it is categorized as a standard vendor resource. Additionally, the skill pulls official benchmark images from Docker Hub, which is expected behavior for its primary function. - [CREDENTIALS_UNSAFE]: The skill requires the
OPENROUTER_API_KEYenvironment variable. This key is used to allow the agent to interact with language model providers via OpenRouter. The management of this key is handled by the platform configuration in the skill's frontmatter. - [DATA_EXFILTRATION]: While the skill accesses external datasets (e.g., from HuggingFace) and interacts with the OpenRouter API, no patterns indicating unauthorized data harvesting or exfiltration were found. The network activity is limited to downloading benchmark data and communicating with LLM endpoints.
Audit Metadata