dspy-best-of-n
Fail
Audited by Gen Agent Trust Hub on May 13, 2026
Risk Level: HIGHREMOTE_CODE_EXECUTIONCOMMAND_EXECUTION
Full Analysis
- [REMOTE_CODE_EXECUTION]: The skill's documentation and code examples (in SKILL.md and examples.md) suggest and implement reward functions that use Python's exec() function to run code strings produced by the language model. This pattern allows an attacker who can influence the AI's output (e.g., via indirect prompt injection) to execute arbitrary Python code on the host system.\n- [COMMAND_EXECUTION]: The use of exec() provides a way to run dynamic commands that can interact with the system environment. The provided examples do not implement a robust sandbox, meaning any code generated by the model is executed with the permissions of the agent process, potentially leading to unauthorized file access or network operations.\n- [DYNAMIC_EXECUTION]: The skill encourages a generate-and-execute workflow for code tasks. This is a significant security anti-pattern as it bypasses static code analysis and relies on the model's safety, which can be subverted.\n- [INDIRECT_PROMPT_INJECTION]: The skill provides an attack surface where instructions embedded in processed data can lead to dangerous code execution.\n
- Ingestion points: The task_description and article article article article article text fields in examples.md and SKILL.md are used to prompt the model to generate content that is later executed or analyzed.\n
- Boundary markers: There are no markers or delimiters used to isolate untrusted content from the instructions.\n
- Capability inventory: The skill uses exec() and dspy.Module calls to process and run generated content.\n
- Sanitization: No sanitization, validation, or structural checks are performed on the generated code strings before they are passed to exec().
Recommendations
- AI detected serious security threats
Audit Metadata