PEFT Fine-Tuning

Overview

Fine-tune large language models efficiently using Parameter-Efficient Fine-Tuning (PEFT) methods. Train 7B to 70B parameter models on consumer GPUs (16-48 GB VRAM) using LoRA, QLoRA, and 25+ adapter methods from the Hugging Face PEFT library. Avoid the cost and hardware requirements of full fine-tuning while achieving comparable results. Written for transformers 5.x, TRL 1.x and PEFT 0.21: several arguments older tutorials use (torch_dtype, warmup_ratio, max_seq_length on the trainer) were renamed or moved.

Instructions

When a user asks to fine-tune a model, determine the approach:

Task A: Set up the environment

pip install torch transformers datasets peft accelerate trl
pip install bitsandbytes   # only for QLoRA; needs an NVIDIA GPU (also AMD/Intel builds exist)
hf auth login            # gated models such as Llama need accepted license + token