Design Product Experiment

A staged rollout is not an experiment. Seal the decision metric and rule before movement. Analyze the assigned cohort. Inconclusive is a legal result; peeking a winner is not.

Open product experiment systems for assignment, guardrails, and readout. Open counterfactual evaluation for logged-policy evaluation.

Use prototype-product when the cheapest discriminating probe is not a randomized assignment. Use analyze-causal-inference when the assignment already happened and must be identified after the fact.