generate-judge-problems
Pass
Audited by Gen Agent Trust Hub on Sep 11, 2026
Risk Level: SAFECOMMAND_EXECUTIONDYNAMIC_EXECUTIONEXTERNAL_DOWNLOADSPRIVILEGE_ESCALATIONINDIRECT_PROMPT_INJECTION
Full Analysis
- [DYNAMIC_EXECUTION]: The skill generates a shell script (
expected.sh) at runtime and executes it via bash to process test cases. It also executes agent-generated model answers (e.g., usingpython3) to produce expected output files, which involves running code derived from user input. - [COMMAND_EXECUTION]: The skill instructs the agent to execute various shell commands, including
bash,python3,bunx, andmise, to automate the problem development and validation lifecycle. - [EXTERNAL_DOWNLOADS]: The skill uses
bunx(npm) to execute the@exercode/problem-utilspackage and usesmise installto download and set up runtimes and command-line tools based on the repository's configuration files. - [PRIVILEGE_ESCALATION]: The instructions include the command
mise trust --yes, which bypasses the manual security prompt for trusting configuration files (mise.toml). This allows the tool to execute potentially unverified configuration tasks without user intervention. - [INDIRECT_PROMPT_INJECTION]: The skill represents a surface for indirect prompt injection as it ingests user-provided source material to generate code that is subsequently executed.
- Ingestion points:
SKILL.mdStep 1 (user-provided learning objectives and source material). - Boundary markers: Absent.
- Capability inventory:
python3(Step 6),bash(Step 6), andbunx(Steps 7, 8, 10). - Sanitization: Absent (relies on agent reasoning and downstream validation tools).
Audit Metadata