Experiment Design Reference Pack

Goal

Support experiment work end to end: turning unknowns into testable hypotheses, screening out work that is not a real experiment, and scoping it so the result is comparable and decision-ready.

Support the step that precedes it as well: translating a stated business outcome into candidate data-science problem classes with the reasoning that produced them, so a practitioner knows what kind of problem is on the table before deciding what to test.

The two concerns stay distinct. Problem-class framing exposes candidates and never selects one. Experiment work assumes a candidate direction already exists and concludes by selecting an experiment with the team.

This pack is general purpose. It applies to data feasibility, architecture, LLM, performance, use-case, UX, prototyping, and hardware experiments, not to data science alone.

Inputs

• The problem statement, customer context, and business driver
• The stated business outcome, when the active concern is problem-class framing
• Known unknowns, assumptions, and risks
• The decision the experiment is meant to unblock
• Prior experiment results, when a sequence of experiments is in flight