grpo-rl-training

Fail

Audited by Gen Agent Trust Hub on Jun 29, 2026

Risk Level: HIGHCOMMAND_EXECUTIONREMOTE_CODE_EXECUTION
Full Analysis
  • [COMMAND_EXECUTION]: The file examples/reward_functions_library.py contains a function run_test_cases that uses the built-in Python exec() function to execute code strings.
  • Evidence: exec(code, exec_globals) inside the run_test_cases function.
  • [REMOTE_CODE_EXECUTION]: The code_execution_reward function in examples/reward_functions_library.py extracts code from model-generated completions and passes it to the unsafe exec() call. Because model outputs are inherently untrusted and can be influenced by input datasets or prompt injections, this creates a direct path for executing malicious code on the system running the training script.
  • Evidence: extracted_code = [extract_code_block(r) for r in responses] followed by passed = run_test_cases(code, test_cases).
  • [INDIRECT_PROMPT_INJECTION]: The skill facilitates the processing of external datasets (e.g., via load_dataset('openai/gsm8k', 'main') in templates/basic_grpo_training.py) which are then used to prompt the model. Malicious data in these datasets could trick the model into generating specific code payloads designed to exploit the unsafe exec() call in the reward library.
  • Ingestion points: get_dataset function in templates/basic_grpo_training.py using the datasets library.
  • Capability inventory: exec() call in examples/reward_functions_library.py and file-saving operations via trainer.save_model().
  • Sanitization: The skill specifically lacks sandboxing for code execution, as noted in the source comments.
Recommendations
  • AI detected serious security threats
Audit Metadata
Risk Level
HIGH
Analyzed
Jun 29, 2026, 12:54 AM
Security Audit — agent-trust-hub — grpo-rl-training