ab-test-setup

Installation
Summary

Structured framework for designing statistically rigorous A/B tests with mandatory validation gates.

  • Enforces hypothesis lock, metric freezing, and sample size calculation before any implementation begins
  • Includes hard gates at three critical points: hypothesis validation, assumptions review, and execution readiness
  • Defines primary, secondary, and guardrail metrics to prevent harmful wins and ensure valid interpretation
  • Covers test type selection, duration estimation, and strict rules against mid-test changes or early stopping
  • Provides refusal conditions and documentation requirements to build institutional learning and avoid repeated test failures
SKILL.md

A/B Test Setup

1️⃣ Purpose & Scope

Ensure every A/B test is valid, rigorous, and safe before a single line of code is written.

  • Prevents "peeking"
  • Enforces statistical power
  • Blocks invalid hypotheses

2️⃣ Pre-Requisites

You must have:

  • A clear user problem
  • Access to an analytics source
  • Roughly estimated traffic volume
Related skills
Installs
514
GitHub Stars
37.3K
First Seen
Jan 19, 2026