Dynamic Few-Shot with dspy.KNN and dspy.KNNFewShot

Guide the user through using DSPy's KNN-based retrieval to dynamically select the most relevant few-shot demonstrations for each input at inference time, rather than using the same static examples for every query.

What KNN and KNNFewShot are

dspy.KNN is an in-memory nearest-neighbor retriever. Given a training set and an embedding function, it converts every training example into a vector. At query time, it embeds the new input, computes dot-product similarity against all stored vectors, and returns the k most similar training examples.

dspy.KNNFewShot is an optimizer (teleprompter) that wraps KNN and BootstrapFewShot together. It compiles a student program so that every forward call first retrieves the k nearest training examples, then uses them as the few-shot demonstrations for the underlying module. The demonstrations change per input -- each query gets the examples most relevant to it.

New input ──> Embed ──> Find k nearest training examples ──> Use as demos ──> Run module

When to use

• Your training examples cover diverse subtasks and you want the LM to see only the most relevant ones for each input (e.g., a classifier that handles many categories, a QA system across different domains)
• Static few-shot examples hurt more than they help because irrelevant demos confuse the model on certain inputs
• You have enough labeled examples (at least 20-50) to make similarity-based retrieval meaningful