Building Fine-Tuning Datasets

A fine-tune is its dataset. Hyperparameters decide whether training converges; the dataset decides what the model becomes. Most disappointing fine-tunes are correctly-configured runs over data that encoded the wrong thing.

Two failures cause most of the damage, and both are settled before a single example is generated: teaching facts that belong in retrieval, and having no way to detect that the model got worse at everything else.

Gate 1: Is this a knowledge problem?

If the goal contains "so it knows our X" — product names, runbook facts, current inventory, policy details — that part is a retrieval problem, and fine-tuning is the wrong tool for it.