experiment-design
Design experiments that actually prove something. Most A/B tests fail because they test vague ideas, run too short, or peek at results. A well-designed experiment has a clear hypothesis, adequate power, and a pre-committed analysis plan.
Hypothesis Template
Every experiment starts with a written hypothesis before any work begins:
"If we [make this specific change] for [this audience], then [this metric] will [change in this direction] by [this amount], because [this reason based on evidence]."
Example:
"If we replace the 5-step onboarding wizard with a single guided first-project flow for new signups, then 7-day activation rate will increase from 23% to 35%, because 4/6 interviewed users said they wanted to 'just start using it' not 'set everything up first.'"
Every part matters:
- Specific change: Not "improve onboarding" — the exact change
- Audience: Who sees this? New users only? Free tier only?
- Metric + direction + amount: A number you'll measure
- Because: The evidence-based reason. No evidence = no experiment.