Optimize Few-Shot Demos with dspy.BootstrapFewShotWithRandomSearch

Guide the user through using DSPy's BootstrapFewShotWithRandomSearch optimizer to find the best set of few-shot demonstrations for their program. This optimizer runs BootstrapFewShot multiple times with different random seeds and keeps the candidate program that scores highest on a metric.

What it is

BootstrapFewShotWithRandomSearch (also known as BootstrapRS) is a prompt optimizer that searches over multiple candidate sets of few-shot demonstrations to find the best one. It wraps BootstrapFewShot and runs it repeatedly with different random subsets of training examples, then evaluates each candidate program on a held-out portion of the trainset.

trainset ──> [ BootstrapFewShot run 1 ] ──> candidate program 1 ──┐
         ──> [ BootstrapFewShot run 2 ] ──> candidate program 2 ──┤
         ──> [ BootstrapFewShot run 3 ] ──> candidate program 3 ──┼──> evaluate all ──> best program
         ──> ...                                                  │
         ──> [ BootstrapFewShot run N ] ──> candidate program N ──┘

How it improves on BootstrapFewShot

BootstrapFewShot runs once: it bootstraps demonstrations from your training data, picks a fixed set, and returns a single optimized program. The result depends heavily on which examples happened to be selected and which traces succeeded. You might get lucky or unlucky.