Generate Synthetic Data

Generate diverse, realistic test inputs that cover the failure space of an LLM pipeline.

Prerequisites

Before generating synthetic data, identify where the pipeline is likely to fail. Ask the user about known failure-prone areas, review existing user feedback, or form hypotheses from available traces. Dimensions (Step 1) must target anticipated failures, not arbitrary variation.

Core Process

Step 1: Define Dimensions

Dimensions are axes of variation specific to your application. Choose dimensions based on where you expect failures.

Dimension 1: [Name] — [What it captures]
  Values: [value_a, value_b, value_c, ...]