The prior argument

The subjective position says the prior is a statement of belief, and its whole point is that it is yours. Head office believed the rate was about 0.12 and had years of chain-wide data behind the belief. Beta(12, 88) wrote that belief down as a hundred pseudo-observations. To write down something weaker because it looks more neutral would be to hide a belief you hold, and hiding a belief is a worse sin than holding a wrong one, because a stated belief can be argued with and a hidden one cannot.

The reference position, sometimes called "let the data speak", says the prior's job is to get out of the way. A flat or nearly flat prior encodes as little as possible, so whatever the posterior says came from the data and not from the analyst. Beta(1, 1) is two pseudo-observations. At Northgate it produced an 89 percent equal-tailed interval from 0.020 to 0.222, and it was the only asserted prior on the bench that contained the truth.

The weakly informative position says both of the others are answering the wrong question. A prior should rule out the absurd, put its weight on the plausible, and be checked before any data arrive by simulating from it: draw rates from the prior, draw eighteen customers at each rate, and look at the redemptions the prior alone predicts. If it predicts fifteen redemptions in eighteen with any regularity, it is a bad prior for a voucher, and you have learned that at no cost. The prior is a regulariser, and regularisers are tested, not believed.

Every one of those paragraphs was written to be as strong as its advocate would make it, and none of them is wrong. That is the situation. Nothing tonight will resolve it, and anybody who tells you it is resolved is telling you which camp they are in.

• Subjective prior. A prior chosen to encode what the analyst or the organisation actually believes, stated as such. Beta(12, 88) at Halloway.
• Reference prior. A flat or near-flat default chosen to encode as little as possible, so the data dominate. Beta(1, 1) is the flat one, and it is still two pseudo-observations.
• Weakly informative prior. A prior that puts weight on plausible values and almost none on absurd ones, chosen to regularise rather than to assert, and checked by prior prediction.
• Prior predictive check. Simulating data from the prior alone, before seeing any real data, to see whether the prior predicts anything absurd. The practitioner track runs one in session p8; tonight you only need to know to ask for it.
• Pseudo-observations. Session 1's currency. For a Beta(a, b) prior, a + b. The number all three positions agree should be stated.
• Likelihood. The model of how the data arise from the rate. Gelman, Simpson and Betancourt's argument is about whether a prior may be chosen with reference to it.
• Learned prior. Session 3's prior, estimated from the other eleven outlets: mean 0.135, worth 116 pseudo-observations. Each position claims it.
• 89 percent interval. As throughout: the 5.5 and 94.5 percent points of the posterior, equal-tailed. Every interval tonight is this kind and says so.

Gelman, Simpson and Betancourt (2017), "The prior can generally only be understood in the context of the likelihood", put the tension in one sentence: "a model encoding true prior information should be chosen without reference to the model of the measurement process, but almost all common prior modeling techniques are implicitly motivated by a reference likelihood." Unpack it. The pure subjective view says your belief about the redemption rate should not depend on how you are going to measure it; a belief is a belief. Yet in practice almost every way people actually choose a prior, including "weakly informative", is shaped by what the measurement can tell them. The third position's own prior predictive check needs a model of eighteen customers to run at all. The first position's purity and the third position's practice pull in opposite directions, and the paper says so.

Their proposed resolution is to place "the choice of prior into the context of the entire Bayesian analysis, from inference to prediction to model evaluation." A prior is judged by what the whole analysis does, not by whether it is a faithful photograph of a belief. Some in the field accept that and treat it as the settled modern view. Others do not: a subjectivist can reply that a prior tuned to make the analysis behave is no longer a prior in the original sense, and a reference-prior advocate can reply that the same move lets an analyst tune toward a conclusion and call it evaluation. Both replies are made, and neither has been withdrawn. The resolution is a position in the argument, not the end of it, and it is presented here as exactly that.

You will rarely hear the labels. You will hear the tells. "We already know roughly what the rate is" is the subjective camp, and the right response is session 1's question: how many observations is that knowledge worth, and where did they come from. "Let's not bias it, just use a flat prior" is the reference camp, and the right response is to ask how wide the resulting interval will be at this sample size and whether anyone can act on that width. "Let's rule out anything above 0.5 and see what the prior predicts" is the weakly informative camp, and the right response is to ask what "plausible" was based on and to see the prior's simulated predictions before agreeing to them.

None of those responses takes a side. Each of them asks the speaker to state their position in a form the other two camps can inspect, and that is the most a leader who is not going to adjudicate the argument can usefully do. It is also, as the next part shows, exactly the ground all three camps already share.

The subjectivist looks at the informed prior's miss and says: the belief was wrong about Northgate, not the act of believing. Head office believed the chain rate applied to every outlet and it did not apply to this one. The remedy is a better belief, and the learned prior is exactly that, a belief about the chain with its source, the other eleven outlets, written down where anyone can challenge it. The reference camp looks at the flat prior's 0.020 to 0.222, the one interval that contained the truth, and says: that width is what honesty looks like at eighteen observations, and the learned prior is not a rival to the data, it is more data, from the outlets next door. The regulariser looks at the learned prior's 0.124, half the error of no pooling, and says: this is what a prior chosen for how the whole analysis behaves looks like, and its pseudo-count of 116 is on the table to be checked. Then, being a regulariser, adds the caveat: 116 is not weak, and Northgate is still under-estimated.

Notice that all three readings are consistent with the numbers. That is what an unsettled argument looks like from the inside: the evidence is shared and the disagreement is about what it means. A leader's job is not to end it. It is to make sure whichever camp is in the room has stated its prior in a form the other two could inspect.

Three things every position accepts, and therefore three things you can require without taking a side. State the prior as pseudo-observations. Beta(12, 88) is a hundred; the learned prior is 116; flat is two. A camp that refuses to give the number is not in the argument. Say where it came from. Head office's history, the other eleven outlets, a deliberate choice to encode nothing. The source is what the other camps will inspect. Simulate from it before seeing data. Draw rates from the prior, draw eighteen customers at each, look at what it predicts. If the picture is absurd, every camp agrees the prior is wrong, and you have found out before the data could hide it. The practitioner track runs that check in session p8; a leader only needs to ask to see it.

Two positions you will never hear a serious person take, and should not credit if a meeting drifts toward them. Nobody argues for a prior that puts zero weight on plausible values. The wrong prior from session 1, Beta(40, 60), was on the bench to show a cost, not because anyone recommends it; a prior that cannot reach the truth at all is a mistake in every camp. And nobody argues that a prior can rescue a bad measurement. If the eighteen vouchers were miscounted, or the redemptions were logged against the wrong outlet, no choice of prior repairs that, and the subjectivist, the reference advocate and the regulariser will all say so in the same breath.

The argument is narrower than it sounds. It is about how much to encode, from where, and whether the encoding may be judged by how the analysis behaves. Keep it there. A meeting that has wandered into "priors are cheating" or "with a good prior the data barely matter" has left the actual argument and joined a worse one.