LLM Fine-Tuning Infrastructure

Train and fine-tune open-source LLMs efficiently — from LoRA on a single GPU to distributed full fine-tuning across multi-node clusters.

When to Use This Skill

Use this skill when:

• Fine-tuning an LLM on domain-specific data (legal, medical, code, support)
• Running QLoRA to fine-tune 70B models on consumer GPUs
• Setting up distributed training with DeepSpeed or FSDP
• Exporting fine-tuned adapters for production serving
• Implementing RLHF, DPO, or instruction tuning pipelines

Prerequisites

Pin a compatible training stack and validate model/dataset permissions and sensitive-data handling. Hardware figures are examples, not capacity guarantees (70B QLoRA may require multiple GPUs/offload). Split held-out evaluation before preprocessing to avoid leakage. External tracking can upload data; disable it unless approved. train.py is not bundled, and the Kubernetes image must be built with Axolotl/dependencies; set output_dir to the mounted /output and bound retries/checkpoint resume to avoid duplicate expensive runs.