simpo-training
Pass
Audited by Gen Agent Trust Hub on Oct 29, 2026
Risk Level: SAFEEXTERNAL_DOWNLOADSCOMMAND_EXECUTIONINDIRECT_PROMPT_INJECTION
Full Analysis
- [EXTERNAL_DOWNLOADS]: The skill fetches the
alignment-handbookfrom Hugging Face's official GitHub repository and refers to datasets hosted on the Hugging Face Hub (e.g., UltraFeedback, HelpSteer). These sources are widely recognized and trusted within the machine learning community. - [COMMAND_EXECUTION]: Provides instructions to perform environment setup, install requirements via
pip, and launch training jobs usingaccelerate launch. These are standard operations for training and fine-tuning language models. - [INDIRECT_PROMPT_INJECTION]: The skill is designed to ingest and process untrusted external data in the form of preference datasets, which could potentially contain malicious instructions intended to influence the model during training.
- Ingestion points: The workflow ingests preference data from the Hugging Face Hub and local JSONL files as described in
references/datasets.md. - Boundary markers: The training documentation does not specify the use of boundary markers or instructions to isolate or ignore embedded prompts within the training data.
- Capability inventory: The skill uses
accelerate launchto execute Python training scripts (scripts/run_simpo.py) and includes examples of using APIs (OpenAI) for synthetic data generation. - Sanitization: While the skill discusses data quality filtering (length, diversity), it does not include explicit sanitization or safety checks for the content of the prompts themselves.
Audit Metadata