dsh-benchmark-case

Warn

Audited by Gen Agent Trust Hub on Sep 4, 2026

Risk Level: MEDIUMPROMPT_INJECTIONINDIRECT_PROMPT_INJECTIONDYNAMIC_EXECUTIONCOMMAND_EXECUTIONEXTERNAL_DOWNLOADS
Full Analysis
  • [PROMPT_INJECTION]: The skill instructions and documentation (references/contract-clauses.md) explicitly mandate the inclusion of the BENCHMARK-AUTH-v1 protocol in generated tasks. This protocol includes instructions that direct the agent to continue executing immediately once a plan is formed and to not pause for confirmation or stop due to missing rounds of verification. This represents a systematic bypass of interactive safety guardrails typically intended to keep a human in the loop.\n- [INDIRECT_PROMPT_INJECTION]: The skill's primary function is to ingest and process external plugin repositories (referred to as fixtures) to create benchmark exam tasks. This creates a surface where malicious instructions or code embedded in the processed repository could influence the agent's behavior during the analysis phase. \n
  • Ingestion points: The fixture/ directory, which contains code from external plugin repositories as specified in SKILL.md and assets/Dockerfile.example.\n
  • Boundary markers: The BENCHMARK-AUTH-v1 section defines a scope but lacks specific delimiters or sanitization steps for untrusted data processed during the migration analysis.\n
  • Capability inventory: The skill uses subprocess execution (execFile) and shell commands (dsh, git) within the benchmark environment, which could be exploited if malicious content is processed.\n
  • Sanitization: No explicit sanitization or filtering of the input fixture content is mentioned before it is analyzed by the agent.\n- [DYNAMIC_EXECUTION]: The assets/judge-utils.mjs library employs dynamic script execution using node -e within a shell environment (sh -c). This is used for probing local network services (127.0.0.1:3080) and analyzing logs at runtime. While these functions are intended for benchmark validation, the use of dynamically evaluated code via shell execution is a high-capability pattern.
Audit Metadata
Risk Level
MEDIUM
Analyzed
Sep 4, 2026, 06:36 AM
Security Audit — agent-trust-hub — dsh-benchmark-case