Allocating effort

Deciding what to work on next is a bandit problem: several options, each with an unknown payoff rate, and every unit of effort spent on one is a unit not spent learning about the others. The two failure modes are symmetric and both are common - committing everything to the current leader before you know it is the leader, and spreading effort evenly long after you do.

Thompson sampling resolves both without tuning, and its allocation has a plain-English meaning: each option gets the share of effort equal to the probability it is the best one.

Workflow