A/B Test Analysis

Quick Start

Analyze experiment results with statistical rigor, including significance testing, power analysis, sample ratio checks, and actionable recommendations for rolling out changes.

Context Requirements

Before analyzing the test, I need:

1. Test Design: What was tested (variants, hypothesis, metric)
2. Test Data: Results from each variant
3. Randomization Unit: User, session, page view, etc.
4. Primary Metric: Success metric for decision-making
5. Guardrail Metrics (optional): Metrics that shouldn't degrade
6. Test Duration: When test started/ended

Context Gathering