trl-training
Installation
SKILL.md
TRL
Each method pairs a *Trainer class with a *Config dataclass. Configs extend transformers.TrainingArguments, so all of its arguments work in any trainer config.
| Trainer | Dataset type |
|---|---|
SFTTrainer |
language modeling or prompt-completion |
DPOTrainer |
preference (chosen/rejected pairs) |
GRPOTrainer |
prompt-only + reward function(s) |
DistillationTrainer |
prompt-only + a teacher model (on-policy distillation) |
KTOTrainer |
unpaired preference (per-sample bool label) |
RewardTrainer |
preference (chosen/rejected pairs); trains a scalar reward model, not a policy |
Many more trainers (OnlineDPO, ORPO, CPO, GKD, …) live in trl.experimental with unstable APIs: https://huggingface.co/docs/trl/experimental_overview
from datasets import load_dataset
from trl import SFTConfig, SFTTrainer