sail-judge · 2026-07-13 · theory companion to Seven Signals and notebooks/thompson-sampling-teaching.jl
Not the data-grounded walkthrough — that's the Pluto notebook. This is the five-minute version: why sail-judge picks an arm the way it does, in theory, before any of the reward-signal story matters.
The name comes from the one-armed bandit — a slot machine. Historically: several machines, each with an unknown, fixed payout rate, and a gambler trying to make money from a limited number of pulls without knowing which machine pays best.
sail-judge doesn't pull levers. Its asker role has three competing constructs it can reach for on any given turn — generation-effect, retrieval-practice, metacognitive-calibration — and its discourse-driver has three more. Every time either role speaks, it has to pick one arm, without knowing in advance which one will actually earn a reward this time.
In the first live run, metacognitive-calibration won its first two draws, succeeded both times, and kept getting picked over and over after that. Was that the bandit being lazy — should it have gone back and tried generation-effect again, just in case? Or was it correctly recognizing a winner and not wasting turns on arms that looked worse?
Both instincts are right some of the time. Favor the arm that's worked (exploit) and you risk missing an arm that would have done better, just because it hasn't had a lucky run yet. Keep sampling every arm evenly (explore) and you waste turns on arms you already have good reason to believe are worse. Every bandit algorithm is a specific answer to how to trade these off — Thompson sampling's answer turns out to be unusually elegant.
Each arm's reward is a coin flip — success or failure, nothing in between. The natural way to represent "how good do I currently believe this coin's success rate is" is the Beta(α, β) distribution: a curve over every possible success rate from 0 to 1, weighted by how plausible each one currently looks.
The update rule is two lines: a success adds 1 to α, a failure adds 1 to β. Start every arm at Beta(1, 1) — flat, "could be anything" — and let real outcomes reshape the curve from there.
Flat means "no opinion" — the curve gives every success rate equal weight. As evidence accumulates the curve both narrows (more confident) and shifts (toward wherever the actual successes landed). That shape — not just a single number like "70% success rate so far" — is the entire trick that makes Thompson sampling work.
The naive approach: track each arm's average success rate so far, and always pick whichever average is currently highest. This is greedy, and it's a trap — an arm that got unlucky early (two failures in its first two tries) looks permanently bad, even though two data points prove almost nothing.
Thompson sampling does something different, and it's exactly what Bandit.choose() implements: draw one random sample from each arm's current Beta posterior, and pick whichever arm's sample came out highest — not whichever arm's average is highest, a fresh random draw from the whole curve, every single time.
An arm with 0/2 successes has a 0.0 average forever, until it's tried again — which greedy will never choose to do.
That same arm is still Beta(1,3) — narrow, low, but not zero. It'll occasionally win a draw anyway, exactly often enough to be worth re-checking, no more.
This is the whole mechanism. Wide, uncertain posteriors produce volatile samples, so under-tried arms occasionally win a draw and get re-tested — automatic exploration. Narrow, confident posteriors produce samples clustered tightly around the true rate, so a genuinely-better arm wins almost every draw — automatic exploitation. No exploration rate to hand-tune, no schedule to decay over time. The uncertainty is the exploration rate, and it shrinks on its own as real evidence comes in.
For this exact problem shape — independent arms, Bernoulli (success/fail) rewards — Thompson sampling has a proven regret bound (Agrawal & Goyal, 2012/2013) matching the theoretical lower bound for the whole problem class. Regret here means the gap between what an algorithm earns and what an oracle who already knew the true best arm would have earned. Thompson sampling's regret grows as slowly as any algorithm's provably can. Nothing cleverer is coming for this exact shape of problem — the sampling mechanism is already about as good as it can be.
which is exactly the catch the rest of this series is about: a mechanism that provably converges as fast as possible will converge just as fast on a bad reward as a good one. See The Question Mark Problem and Seven Signals for that half.