Duolingo Experimentation — Map of Content

Duolingo runs hundreds of experiments per week and treats the result as the decision, not as one input. The handbook calls this Show Don't Tell: when people disagree, the experiment is the answer, not the more senior person's intuition. This skill packages that protocol.

This skill is structured as a graph: scan the descriptions below, follow only the [[wikilinks]] you need.

The protocol

• [[references/show-dont-tell]] — the principle: metrics decide, not titles.
• [[references/hypothesis-design]] — what makes a testable hypothesis vs. a wish.
• [[references/ab-test-structure]] — variant/control, randomization unit, exposure rules.

Metrics

• [[references/metric-selection]] — picking a primary metric that actually maps to long-term retention, not the easiest-to-move one.
• [[references/guardrail-metrics]] — the metrics that must not go down, even if the primary does up.
• [[references/sample-size]] — how big, how long; the cost of stopping early.

Pitfalls