llm-training
Installation
SKILL.md
LLM Training
Frameworks and techniques for training and finetuning large language models.
Framework Comparison
| Framework | Best For | Multi-GPU | Memory Efficient |
|---|---|---|---|
| Accelerate | Simple distributed | Yes | Basic |
| DeepSpeed | Large models, ZeRO | Yes | Excellent |
| PyTorch Lightning | Clean training loops | Yes | Good |
| Ray Train | Scalable, multi-node | Yes | Good |
| TRL | RLHF, reward modeling | Yes | Good |
| Unsloth | Fast LoRA finetuning | Limited | Excellent |