Experiment Sensitivity Optimization

Use this skill to redesign an experiment so it can detect meaningful effects with fewer users, less time, or clearer metrics. It focuses on minimum detectable effect, metric sensitivity, capping, variant reduction, CUPED, and variance reduction.

Source Traceability

Primary source: Next-Level A/B Testing by Leemay Nassery. Guidance is transformed and paraphrased from Chapter 3 on experiment design, sensitive metrics, minimum detectable effect, capping, reducing variants, and CUPED; and Chapter 6 on stratified random sampling and covariate adjustments.

Related skills:

• ab-test-design-brief for baseline experiment specs.
• trustworthy-experiment-insights for judging whether a result is believable.
• experimentation-throughput-strategy for capacity and test scheduling.