Fine-Tune LM Weights with dspy.BootstrapFinetune

Guide the user through using DSPy's BootstrapFinetune optimizer to automatically generate training data from successful reasoning traces and fine-tune a language model's weights. This is the heaviest optimization DSPy offers -- it changes the model itself, not just the prompt.

What is BootstrapFinetune

dspy.BootstrapFinetune is an optimizer that tunes LM weights rather than prompts. It works in two phases:

1. Bootstrap: Run your program on every training example, keep the traces where your metric passes.
2. Fine-tune: Send those successful traces to the model provider's fine-tuning API (or a local training loop) and train the model weights on them.

The result is a version of your program backed by a fine-tuned model that has internalized the reasoning patterns from the bootstrapped traces.

Training examples ──> Run program ──> Keep passing traces ──> Fine-tune model weights

When to use BootstrapFinetune