Adaptive Experimentation Strategy

Use this skill to decide whether and how to use adaptive testing strategies instead of fixed-horizon A/B tests. It covers sequential testing, multi-armed bandits, Thompson sampling, contextual bandits, exploration/exploitation, and the data and engineering readiness required to operate them.

Source Traceability

Primary source: Next-Level A/B Testing by Leemay Nassery. Guidance is transformed and paraphrased from Chapter 7 on adaptive testing, sequential testing, multi-armed bandits, Thompson sampling, contextual bandits, and engineering requirements.

Related skills:

• experiment-type-selection for choosing simpler experiment types first.
• ml-experiment-evaluation for model and ranking evaluation paths.
• experiment-verification-monitoring for operational health and alerting.