A/B Testing

Overview

A/B Testing brings experimental rigor to GTM optimization. The core principle: most "optimizations" are guesswork dressed as strategy. Teams change subject lines because someone "feels" they're better, rewrite copy based on a single positive reply, or abandon channels because of one bad week. This skill prevents the waste of optimizing based on noise rather than signal.

The non-obvious rule: statistical significance is necessary but insufficient. A statistically significant 10% improvement in open rate that produces 0% more meetings is a distraction. Optimize for downstream revenue metrics — reply rate, meeting rate, conversion rate — not vanity metrics. Every test must trace its impact through the full funnel.

This skill produces: an Experiment Design Document with hypothesis, success metrics, power analysis, and methodology; a prioritized ICE/PIE test backlog; a 30/60/90-day testing roadmap; and a Results Analysis Report with statistical tests, confidence intervals, and scale/stop/kill recommendations.

When to Use

• User says "A/B test" or "split test" → activate this skill
• User mentions "test subject lines" or "test email copy" → use for experimental design
• User asks "is this statistically significant" or "what sample size do I need" → activate for statistical guidance
• User says "experiment design" or "test plan" → use this skill
• User asks "how do I prioritize tests" or "what should I test next" → use ICE/PIE scoring
• User mentions "testing roadmap" or "experiment cadence" → plan the roadmap
• Trigger phrases: A/B testing, split testing, experiment, statistical significance, confidence interval, sample size, test variant, control group, multivariate test