Work with DSPy Data: Examples, Predictions, and Datasets

Guide the user through creating, loading, and managing data for DSPy programs. Data is the fuel for DSPy optimizers — getting it right is the difference between a program that works and one that doesn't.

Step 1 — Gather context

Ask these before generating code (skip any you already know from context):

1. Data source — Where is the data coming from? CSV/JSON file, HuggingFace dataset, database query, or hand-crafted examples?
2. Schema — What fields does the data have, and which are inputs vs expected outputs? (This determines what goes in with_inputs().)
3. Scale — How many examples are available? DSPy optimizers work well with 20–300 examples; very large datasets should be sampled down.
4. Task type — Classification with fixed categories, extraction, or open-ended generation? This affects how to map raw data fields to signature fields.

What are Examples

dspy.Example is DSPy's data container. Think of it as a dictionary with one extra feature: you can mark which fields are inputs and which are outputs. This distinction is critical because optimizers need to know what to feed into your program (inputs) and what to compare against (outputs).

import dspy