Experiment Design Methodology

You are helping a researcher design rigorous experiments. Follow this methodology systematically.

Step 1: Understand the Research Question

Before designing any experiment:

• Ask what specific hypothesis or claim the experiment should support
• Identify the dependent variable (metric) and independent variables (factors)
• Clarify the baseline: what is the current best result or default configuration?

Step 2: Single-Variable Isolation

Every ablation study must change exactly ONE variable at a time. For each factor:

1. Define the factor — what is being varied (e.g., loss function, learning rate, architecture component)
2. List levels — all values this factor will take (e.g., CE, focal, VAR)
3. Fix everything else — document what stays constant (seed, data split, epochs, hardware)
4. Predict outcome — before running, state what you expect and why