A/B testing

A test only answers the question it was designed for: a hypothesis written before launch, one primary metric, a sample size and stopping rule fixed in advance, and an analysis that reads the p-value and interval for what they are. This skill is the tool-agnostic method, from [coreyhaines31/marketingskills, ab-testing] (MIT, (c) 2025 Corey Haines, copied with its notice). Its sample-size tables are replaced by values computed from the standard two-proportion formula (see SAMPLE-SIZE.md). Running a test in PostHog (flags, exposures, its statistics engine and health checks) is posthog EXPERIMENTS.md and the files it links. Test ideas for a page are cro SKILL.md; variant copy is anti-slop-copy SKILL.md. Store experiments are aso SKILL.md; paywall tests are app-growth SKILL.md.

Product context: read .agents/product-context.md first if it exists (defined in anti-slop-copy CONTEXT.md) and ask only for what it doesn't cover. The source names this file .agents/product-marketing.md.

Apply only the routes and checks for the selected job. Read a linked skill's named file from its installed folder when available, or from the linked source. Report an unavailable required reference instead of inventing it. [skill-authoring, DISTRIBUTION.md#composing-skills]

The job	Read
The required inputs, a sample-size table computed from the formula, duration with equal and unequal splits, minimum and maximum duration, multiple variants, too-high samples, sequential testing, calculators, the can-I-run-this check	SAMPLE-SIZE.md
Test plan, results write-up, test repository row, quick brief, stakeholder update, prioritization scorecard, hypothesis bank, playbook entry	TEMPLATES.md
The experiment loop, hypothesis sources, ICE scoring, velocity, cadence, common mistakes, the questions to ask	PROGRAM.md
Running it in PostHog: plan, rollout and split, metrics, running time, statistics, health checks	posthog EXPERIMENTS.md
Screening headlines in ads, then proving the winner on the landing page with a fresh randomized test	paid-ads CAMPAIGN-PLANNING.md

Before designing

[coreyhaines31/marketingskills, ab-testing] Understand three things first: