LLM Fine-Tuning

Overview

Fine-tuning adapts a base LLM to a task or style. For almost all practitioners the right default is parameter-efficient fine-tuning (LoRA/QLoRA) — it trains <1% of weights, fits on a single GPU, and avoids catastrophic forgetting. Reach for full fine-tuning only with strong justification and budget.

When to use

• A base/instruct model is close but needs domain style, format, or behavior.
• You have a few hundred to tens of thousands of quality examples.

First decide: do you even need to fine-tune?

Try prompting + few-shot + RAG first (see the rag-pipeline skill). Fine-tune when you need consistent format/style, lower latency/cost than long prompts, or to internalize a large example set.

Method selection