Unsloth — Fast LoRA and QLoRA Fine-Tuning for Open LLMs

Overview

Unsloth is an open-source training library (plus a desktop app and web UI called Unsloth Studio) that patches Hugging Face Transformers, PEFT and TRL with hand-written Triton kernels. The same LoRA or QLoRA run needs noticeably less VRAM and finishes faster, which is what makes an 8B model trainable on a 16 GB card or a free Colab T4. There are two code entry points:

• Python API (the Apache-2.0 core library): FastLanguageModel.from_pretrained loads a model (4-bit by default), FastLanguageModel.get_peft_model attaches LoRA adapters, and a normal TRL SFTTrainer runs the training.
• unsloth CLI (runs the Studio training backend, AGPL-3.0): unsloth train runs the same training from flags or a YAML file, and unsloth export turns a checkpoint into merged weights, GGUF or a LoRA adapter.

After training, Unsloth saves in the formats people deploy: GGUF files (plus an Ollama Modelfile when it knows the chat template), merged 16-bit weights for vllm serve, or a small LoRA adapter.

Absolute minimum VRAM from the Unsloth requirements page: 8B QLoRA about 6 GB, 14B about 8.5 GB, 70B about 41 GB; 16-bit LoRA needs roughly three to four times more.

Instructions

Installation

Install into a fresh virtual environment. uv picks the right PyTorch build for the installed CUDA driver. The bare unsloth package covers only the Python API; the unsloth train/export/list-checkpoints commands import the Studio backend (FastAPI, uvicorn, gguf and more), which comes with the studio extra: