grpo-rl-training
Fail
Audited by Gen Agent Trust Hub on Jun 29, 2026
Risk Level: HIGHCOMMAND_EXECUTIONREMOTE_CODE_EXECUTION
Full Analysis
- [COMMAND_EXECUTION]: The file
examples/reward_functions_library.pycontains a functionrun_test_casesthat uses the built-in Pythonexec()function to execute code strings. - Evidence:
exec(code, exec_globals)inside therun_test_casesfunction. - [REMOTE_CODE_EXECUTION]: The
code_execution_rewardfunction inexamples/reward_functions_library.pyextracts code from model-generated completions and passes it to the unsafeexec()call. Because model outputs are inherently untrusted and can be influenced by input datasets or prompt injections, this creates a direct path for executing malicious code on the system running the training script. - Evidence:
extracted_code = [extract_code_block(r) for r in responses]followed bypassed = run_test_cases(code, test_cases). - [INDIRECT_PROMPT_INJECTION]: The skill facilitates the processing of external datasets (e.g., via
load_dataset('openai/gsm8k', 'main')intemplates/basic_grpo_training.py) which are then used to prompt the model. Malicious data in these datasets could trick the model into generating specific code payloads designed to exploit the unsafeexec()call in the reward library. - Ingestion points:
get_datasetfunction intemplates/basic_grpo_training.pyusing thedatasetslibrary. - Capability inventory:
exec()call inexamples/reward_functions_library.pyand file-saving operations viatrainer.save_model(). - Sanitization: The skill specifically lacks sandboxing for code execution, as noted in the source comments.
Recommendations
- AI detected serious security threats
Audit Metadata