landing-page-experimentation
landing-page-experimentation
Assign variants on the server, respect the sample-size floor, and distrust the numbers until the bots are filtered.
The failure this fixes
Teams bolt a client-side A/B script onto a page; it swaps the DOM after render (flash of original content, 200-500ms delay), and the anti-flicker snippet that hides the body to mask it directly tanks LCP/CLS — so the test suppresses the conversion it measures. Then they run tests that can never reach significance (only ~13% of tests produce a winner; low-traffic pages can't clear the ~300 conversions/variant floor), and trust analytics corrupted by bot/AI traffic (0-second sessions, some ad-driven tests >98% bots) and ad-blockers that eat 10-30% of client events. The deepest scar: tooling isn't the lever — the same AI CRO tools deliver a few % lift DIY vs several-fold more expert-guided, because human hypothesis quality is what moves the number.