Experiment Design

Design rigorous machine learning experiments that produce credible, reproducible results.

Design Checklist

Work through each section before writing any code.

1. Hypothesis

State the hypothesis as a falsifiable claim:

"We claim that [method X] achieves [metric Y] on [dataset Z] because [mechanism]."

If the hypothesis is vague, help the user sharpen it before proceeding.

2. Independent and Dependent Variables

• Independent variable: What is being changed (e.g., architecture, loss function, data augmentation)?
• Dependent variable: What is being measured (e.g., accuracy, FID score, wall-clock time)?
• Controlled variables: List everything held constant.