Same data, four beliefs
A prior is not a setting to get right. It is a claim about the rate before the data arrive, and like any claim it can be worth a little or a lot and it can be right or wrong. Session 1 gave the unit: a Beta(a, b) prior is worth a + b pseudo-observations. Tonight puts four priors of different worth and different centre on the same eighteen customers, and because the truth is known, each one gets a score rather than a defence.
★ What you walk out with tonight. The Northgate table: four priors, four posterior means, four 89 percent intervals, one truth, and only the flat prior's interval containing it. The arithmetic of why: an informed prior worth 100 pseudo-observations gave the eighteen real customers a weight of 18 / (18 + 100) = 0.15, and Northgate's truth sat outside what the prior allowed. And the Westfield result that ends "the data will wash it out": at n = 900 a wrong prior worth 100 still moves the posterior mean by 0.026. Data outvote a prior; they do not wash it out.
LiveThe four priors on the bench4 min▶
Flat is Beta(1, 1): mean 0.500, two pseudo-observations, one of each. It says almost nothing and it is not nothing. Weak is Beta(2, 14): mean 0.125, sixteen pseudo-observations, a gentle lean toward the chain's neighbourhood. Informed is Beta(12, 88): mean 0.120, a hundred pseudo-observations, the chain-wide rate held with real conviction. Wrong is Beta(40, 60): mean 0.400, also a hundred pseudo-observations, held with the same conviction and centred somewhere no Halloway outlet has ever been.
The two hundred-observation priors are the interesting pair. They have identical weight against the data and opposite relationships to the truth, so between them they separate the two things a prior can cost you: its weight, which is how much of the answer it decides, and its distance, which is how wrong that share of the answer is. Weight is arithmetic and falls with n. Distance is a fact about the world and does not fall at all.
Halloway's analyst ran Northgate under Beta(12, 88) because the chain had sat near 0.12 for years, and reported 0.110 with an interval of 0.068 to 0.159. It was the tidiest number in the deck. Northgate's true rate is 0.192. Nobody had asked whether the newest outlet, in a different kind of neighbourhood, was the kind of outlet the hundred pseudo-observations had been drawn from.
Self-studyWhat each prior is actually claiming3 min▶
- Flat, Beta(1, 1). "Any rate from 0 to 1 is equally plausible." Two votes. The posterior will be almost entirely the data, which at n = 18 means almost entirely noise.
- Weak, Beta(2, 14). "Probably somewhere around 0.125, but I would not bet much." Sixteen votes against Northgate's eighteen, so the data carry 18 / 34, a little over half.
- Informed, Beta(12, 88). "Around 0.120, and I have a hundred customers' worth of reasons to think so." The data carry 18 / 118 = 0.15. The prior decides 85 percent of the answer before a single voucher is counted.
- Wrong, Beta(40, 60). "Around 0.400, with the same hundred customers' worth of confidence." Same 0.15 data weight as informed. The difference between the two posteriors is entirely distance.
- Pseudo-observations, a + b. The single number to ask of any prior. It is in the same units as n, so the comparison is immediate: is the belief worth more or less than the evidence?
- Data weight, n / (n + a + b). The share of the posterior mean the evidence decides. 0.90 for flat at Northgate, 0.15 for either hundred-observation prior, 0.90 for either of them at Westfield.
- Residual bias. Posterior mean minus observed rate. What the prior is still doing to the answer after the data have voted. At Westfield it runs from -0.0021 to +0.0259.
Four posteriors, one truth, three misses
9 min liveThe same one redemption in eighteen, four times over. Each posterior is a Beta with the counts added, exactly as in session 1, so the four curves below are Beta(2, 18), Beta(3, 31), Beta(13, 105) and Beta(41, 77). The truth is 0.192. Look at which curves put any weight there.
LiveThe informed prior was confidently wrong5 min▶
Here is the table, and it is the one to remember.
| Prior | Beta(a, b) | Prior mean | Pseudo-obs | Northgate post. mean | 89% interval | Contains 0.192? |
|---|---|---|---|---|---|---|
| Flat | (1, 1) | 0.500 | 2 | 0.100 | 0.020 to 0.222 | yes, and it is wide |
| Weak | (2, 14) | 0.125 | 16 | 0.088 | 0.026 to 0.175 | no |
| Informed | (12, 88) | 0.120 | 100 | 0.110 | 0.068 to 0.159 | no |
| Wrong | (40, 60) | 0.400 | 100 | 0.347 | 0.279 to 0.419 | no |
The informed prior is the row that matters. It was a hundred pseudo-observations centred on 0.120 against eighteen real ones, so the data carried weight 18 / (18 + 100) = 0.15 and the prior decided the other 0.85. Northgate is the one outlet whose true rate, 0.192, sits outside what that prior allowed for, and so the posterior landed at 0.110 with an interval of 0.068 to 0.159 that does not reach the truth. That is the cost of a prior, measured. The prior was built from a true fact about the chain and applied to the one outlet unlike the chain, and no amount of care in choosing it would have changed the arithmetic: 0.15 of the answer was ever going to come from Northgate's own customers.
The flat prior's interval contains the truth, and it is wide enough to be nearly useless for any decision: 0.020 to 0.222 covers a rate that would close the outlet and a rate that would make it the best in the chain. So the table does not say "use flat priors". It says that the interval which reads like knowledge is the one that missed, and that the question to ask of a narrow interval is how many pseudo-observations bought the narrowness and where they came from.
Self-studyWeight against distance, in numbers3 min▶
Informed and wrong have the same weight at Northgate, 0.15 for the data and 0.85 for the prior, so the gap between their posteriors is entirely the gap between their centres. Informed sits at 0.120 and lands at 0.110. Wrong sits at 0.400 and lands at 0.347. Both are 85 percent belief; one belief was 0.072 from the truth and the other was 0.208 from it. The posterior is only ever as good as the smaller of two things: how much of it the data decided, and how close the prior was on the part they did not.
The weak prior is the instructive middle case. Sixteen pseudo-observations against eighteen gives the data 18 / 34, a little over half, and the posterior mean of 0.088 lands between the observed 0.056 and the prior's 0.125, closer to the observed side. Its interval, 0.026 to 0.175, misses the truth by 0.017. A prior can be modest, sensible and still wrong on the one outlet that matters, and modesty only reduces the cost, it does not remove it. Session 6 shows the prior that does best here, and it is not chosen at all: it is learned from the other eleven outlets.
Data outvote a prior. They do not wash it out.
6 min liveWestfield is Halloway's flagship: 900 vouchers, 127 redeemed, observed 0.141, true rate 0.150. Fifty times Northgate's volume, and the standard reassurance is that at this size the prior no longer matters. Run the same four priors and read the posterior means: flat 0.142, weak 0.141, informed 0.139, wrong 0.167.
LiveThe Westfield arithmetic4 min▶
Residual bias is the posterior mean minus the observed rate, and it is what the prior is still doing after 900 customers have voted. Flat: +0.0008. Weak: -0.0003. Informed: -0.0021. Wrong: +0.0259. The first three are within a few thousandths of zero and of each other; at this volume, a sensible prior of any weight is nearly invisible. The fourth is thirty times larger than any of them, and it has exactly the same weight as the informed prior: 100 / (900 + 100) = 0.10 for the prior, 0.90 for the data.
So a prior worth 100 pseudo-observations still moves the answer by 0.026 at n = 900, and the reason is that the weight fell to a tenth while the distance stayed where it was. 0.400 is 0.259 from the observed 0.141, and a tenth of 0.259 is 0.026. Data outvote a prior; they do not wash it out. A hundred votes take a long time to outvote, and on any dataset you will actually have, the prior's tenth or twentieth of the answer is still there, spending itself on whatever it was centred on.
Halloway's finance team took the Westfield figure as the safe one, on the grounds that 900 was plenty. Under the informed prior it very nearly was: 0.139 against an observed 0.141. Had the 0.400 belief a rival chain's numbers suggested been used instead, the flagship estimate would have carried 0.026 of that belief into the chain-wide budget, and 900 would still have sounded like plenty.
Self-studyWhich prior, then?3 min▶
Tonight scores priors; it does not settle which to choose. Whether a prior should encode belief, stay out of the way so the data speak, or regularise gently toward plausible values and be checked by what it predicts is a live disagreement in the field, and leader session 5 takes it up as the argument it is, with Gelman, Simpson and Betancourt's resolution. Nothing here decides it.
What the two tables do settle is what any choice has to answer for. Before a prior goes into a model, its pseudo-count should be stated next to n, so the data weight is a number in the room. Its centre should be stated next to where the data land, so the distance is visible. And the case in front of you should be compared to the cases the prior was built from, because Northgate is what happens when it is not. Session 6 shows the prior that answers all three by construction: one learned from the other eleven outlets, whose weight the data set themselves.
Loop the four priors over two outlets
20 min build-alongStill scipy.stats.beta and nothing else. The whole exercise is one nested loop: for each outlet, for each prior, add the counts and read off the mean, the two quantiles, and whether the truth is inside. Print it as a table and compare every row to the ones above.
Set the bench up. Four priors as (a, b) pairs, two outlets as (n, y, truth), and the 89 percent equal-tailed quantiles from session 1.
from scipy.stats import beta
PRIORS = {
"flat": (1, 1), # mean 0.500, 2 pseudo-obs
"weak": (2, 14), # mean 0.125, 16 pseudo-obs
"informed": (12, 88), # mean 0.120, 100 pseudo-obs
"wrong": (40, 60), # mean 0.400, 100 pseudo-obs
}
OUTLETS = {
"Northgate": (18, 1, 0.192), # n, redeemed, true rate
"Westfield": (900, 127, 0.150),
}
LO, HI = 0.055, 0.945 # 89 percent equal-tailed, the ArviZ 1 default
Northgate under all four. Add the counts, print the posterior mean and the interval, and test whether 0.192 lies inside it. Four rows, one True.
n, y, truth = OUTLETS["Northgate"]
for name, (a, b) in PRIORS.items():
post = beta(a + y, b + n - y)
lo, hi = post.ppf(LO), post.ppf(HI)
print(f"{name:9s} pseudo={a + b:3d} mean={post.mean():.3f} "
f"89%={lo:.3f} to {hi:.3f} contains={lo <= truth <= hi}")
# flat pseudo= 2 mean=0.100 89%=0.020 to 0.222 contains=True
# weak pseudo= 16 mean=0.088 89%=0.026 to 0.175 contains=False
# informed pseudo=100 mean=0.110 89%=0.068 to 0.159 contains=False
# wrong pseudo=100 mean=0.347 89%=0.279 to 0.419 contains=False
Put the data weight beside each row. n / (n + a + b). This is the number that explains the table: the two hundred-observation priors gave Northgate's own customers 0.15 of the answer.
for name, (a, b) in PRIORS.items():
print(f"{name:9s} data weight at Northgate = {n / (n + a + b):.2f}")
# flat data weight at Northgate = 0.90
# weak data weight at Northgate = 0.53
# informed data weight at Northgate = 0.15
# wrong data weight at Northgate = 0.15
Westfield under all four. Same loop, 900 vouchers. Print the posterior mean, the residual bias against the observed 0.141, and the data weight. Read the wrong row twice.
n, y, truth = OUTLETS["Westfield"]
observed = y / n # 0.141
for name, (a, b) in PRIORS.items():
post = beta(a + y, b + n - y)
print(f"{name:9s} mean={post.mean():.3f} "
f"bias={post.mean() - observed:+.4f} "
f"data weight={n / (n + a + b):.3f}")
# flat mean=0.142 bias=+0.0008 data weight=0.998
# weak mean=0.141 bias=-0.0003 data weight=0.983
# informed mean=0.139 bias=-0.0021 data weight=0.900
# wrong mean=0.167 bias=+0.0259 data weight=0.900
Check the distance rule by hand. The wrong prior's bias should be its weight times its distance from the observed rate. It is, to the fourth decimal.
a, b = PRIORS["wrong"]
w_prior = (a + b) / (n + a + b) # 100 / 1000 = 0.100
distance = a / (a + b) - observed # 0.400 - 0.141 = 0.259
print(f"weight x distance = {w_prior * distance:+.4f}") # +0.0259
Write the two sentences. One for each outlet, in the form session 9 will insist on. "At Northgate, under a Beta(12, 88) prior worth 100 pseudo-observations, the 89 percent equal-tailed interval is 0.068 to 0.159; the data weight is 0.15." "At Westfield the same prior gives 0.139 against an observed 0.141; a Beta(40, 60) prior of the same weight gives 0.167." Both sentences name the prior's weight. Neither hides behind n.
Change one thing at a time. The loop holds the data fixed and varies the prior. Session 6 does the opposite: it holds a family of priors fixed and lets twelve outlets' data choose the one weight, kappa, that fits them all. Keep tonight's loop; you will run it again there with the learned prior as a fifth row.
Three questions
The informed prior Beta(12, 88) gave Northgate an interval of 0.068 to 0.159 against a truth of 0.192. Which statement describes what happened?
0.120 is close to the chain-wide rate; the prior encoded a true fact about the chain. The failure was weight against a case unlike the population the prior came from. The flat prior's interval, from the same eighteen observations, did contain the truth, so C is wrong on the evidence.
At Westfield, n = 900, the informed and wrong priors both carry data weight 0.90. Their residual biases are -0.0021 and +0.0259. Why the thirty-fold difference?
Both are Beta priors with a + b = 100, so their weights are identical. Bias is weight times distance: 0.10 x 0.259 = 0.0259. Volume shrinks the weight for both equally and does nothing to either distance.
A colleague reads the Northgate table and concludes that flat priors are the safe choice. What does the table actually support?
The flat prior is two pseudo-observations at 0.500, which is an assumption, and its interval spans closure-worthy and best-in-chain rates at once. The weak prior missed by 0.017; the informed by 0.033. Missing by less is not containing. The question of which kind of prior to choose is leader session 5's, and it is not settled here.
Before session 3
30 minRun tonight's loop on Old Quay: 31 vouchers, 1 redeemed, observed 0.032, true rate 0.090. Report the four posterior means, the four intervals, which contain 0.090, and the data weight on each row. Then write one sentence on whether Old Quay looks like Northgate or like Westfield, and why. Session 3 stops relying on the Beta formula and builds the same posterior from a grid and then from samples, so that when there is no closed form you can still get the curve.