A/B Testing for E-commerce

Overview

A/B testing (split testing) runs controlled experiments where a random subset of visitors sees a variant while the rest see the control. Statistical analysis then determines whether any difference is real or due to chance. Good testing disciplines — calculating required sample size before starting, running tests for at least two full weeks, and never stopping early — separate genuine insights from noise.

This skill guides you through running A/B tests on your specific platform, choosing the right tools, and interpreting results correctly.

When to Use This Skill

• When making product page, checkout, or pricing changes and wanting data-driven validation
• When migrating from a client-side A/B testing tool to server-side assignment for accuracy
• When needing statistical power calculations before starting an experiment
• When analyzing experiment results and determining when to ship or kill a variant
• When running a pricing test and needing to ensure consistent pricing per customer
• When wanting to understand what sample size is needed before a test is meaningful

Core Instructions