Contract

• Input: feature change, user segment, metric of interest.
• Output: test design with power analysis and rollback rules.
• Side effects: none (design only; execution requires deployment and user exposure).
• Dependencies: analytics platform (for measurement), experiment platform (for assignment).
• Stop condition: design complete; power computed; rollback rules defined.
• Risk: medium — bad test design leads to false conclusions; exposure to users requires ethical review.
• Boundary: designs experiment; does not expose users to unapproved changes.

A/B Testing Design

Design an A/B test — hypothesis, randomisation, metrics, statistical power — with valid inference.

Process