openrlhf-training

Fail

Audited by Gen Agent Trust Hub on Jun 29, 2026

Risk Level: HIGHCOMMAND_EXECUTIONREMOTE_CODE_EXECUTIONPROMPT_INJECTION
Full Analysis
  • [COMMAND_EXECUTION]: The installation instructions in SKILL.md suggest running Docker with the --cap-add=SYS_ADMIN flag. This grants the container significant privileges on the host system, increasing the risk of container escape.
  • [COMMAND_EXECUTION]: The SKILL.md file advises using sudo pip uninstall for resolving package conflicts. Executing package managers with root privileges is a security risk as it allows for arbitrary modifications to the host system during the uninstall process.
  • [COMMAND_EXECUTION]: In references/custom-rewards.md, the example reward function uses subprocess.run to execute Python code via pytest. This pattern allows the execution of system commands based on the content of a temporary file containing model-generated text.
  • [REMOTE_CODE_EXECUTION]: The skill framework facilitates the loading and execution of arbitrary Python logic via parameters like --remote_rm_url and --agent_func_path. This enables the execution of user-supplied or remotely referenced code within the training environment.
  • [PROMPT_INJECTION]: The skill architecture is susceptible to indirect prompt injection. Untrusted data from model outputs (queries) and external labels are processed by reward functions and agents. If these scripts perform actions like code execution (as shown in the documentation examples), a malicious model response could lead to unintended command execution.
  • Ingestion points: queries, prompts, and labels parameters in reward_func.py and agent_func.py referenced in references/custom-rewards.md.
  • Boundary markers: None provided in the implementation templates to distinguish data from executable instructions.
  • Capability inventory: subprocess.run calls documented in references/custom-rewards.md.
  • Sanitization: None; the provided example directly writes generated code to a file and executes it via pytest.
Recommendations
  • AI detected serious security threats
Audit Metadata
Risk Level
HIGH
Analyzed
Jun 29, 2026, 12:55 AM
Security Audit — agent-trust-hub — openrlhf-training