A/B Testing

This skill designs and analyzes A/B tests with statistical rigor. It covers hypothesis formulation, metric selection, sample size determination, test implementation, significance evaluation, and actionable result interpretation. Every test is designed to produce reliable, trustworthy conclusions.

When to Use This Skill

• Designing A/B tests for websites, apps, or emails
• Framing clear, testable hypotheses
• Selecting primary and guardrail metrics
• Calculating required sample sizes and test duration
• Implementing tests with proper randomization
• Evaluating statistical significance and confidence intervals
• Analyzing test results to make data-driven decisions
• Determining whether to ship, iterate, or abandon a change
• Planning multi-variant tests

What This Skill Does