Bayes’ Rule, Priors, Likelihoods, and Posteriors

Bayes’ rule connects P(θ|y) to P(y|θ), but it does not make the two quantities interchangeable. The likelihood describes how plausible the observed data are under each parameter value. The prior describes parameter uncertainty before the current data are incorporated. Their product, after normalization, gives the posterior. Inference therefore depends on both a data model and prior assumptions. Calling a posterior 'what the data say' hides part of the calculation.

The rule was printed by someone else. Its author was already dead when it reached the Royal Society, and the title page says so in full: “An Essay towards Solving a Problem in the Doctrine of Chances. By the Late Rev. Mr. Bayes, F. R. S. Communicated by Mr. Price, in a Letter to John Canton, A. M. F. R. S.” Richard Price sent it. Philosophical Transactions of the Royal Society of London carried it in 1763.

The denominator makes the result a probability distribution over parameters. The sequence is fixed. Represent uncertainty before the current dataset as the prior p(θ). Describe how the data would be generated for each θ as the likelihood p(y|θ). Multiply the two to form the unnormalized posterior p(y|θ)p(θ). Divide by the evidence p(y) to obtain p(θ|y). Then average future outcomes over that posterior, rather than over a single selected parameter value.

The numbers below are not a hypothetical. At a continuing-education session in 2007 a room of practising specialists was given a prevalence of 1%, a sensitivity of 90% and a false-positive rate of 9%. “I asked 160 gynecologists”, Gerd Gigerenzer writes. The question had four options. The correct one is that about 1 in 10 women with a positive mammogram has cancer. Only 21% chose it, worse than the 25% expected from guessing. The majority answered 81% or 90%, which is to say they answered with the sensitivity. Gigerenzer and four co-authors report it in Psychological Science in the Public Interest in 2007: “The number of physicians who found the best answer, as documented in medical studies, was slightly less than chance (21%).” Steven D. Stovitz, reproducing the item independently, records the same tally — “only 21% (34 out of 160) answered C, the correct answer”.

The fix is a change of representation, not of arithmetic. Restated as natural frequencies, Gigerenzer reports, “When natural frequencies were used, 87% of the gynecologists saw that the best estimate was 10 in 1,000.”

Nor is this one unlucky room. The classic reversal question — prevalence 1/1000, false-positive rate 5%, correct answer about 2% — went to 60 physicians, house officers and students at Harvard teaching hospitals. Casscells and two co-authors found 11 of 60 correct (NEJM 1978). As Stovitz puts it, “Only 18% answered the open-ended question correctly.” Twenty-seven of the 60 answered 95%, the complement of the false-positive rate. Manrai and four co-authors re-ran the identical question at a Boston-area hospital in July 2013. They found 14 of 61 (23%) correct, and the most common answer, from 27 of 61 (44%), was again “95%” (JAMA Internal Medicine, 2014). Thirty-five years apart, the same wrong number.

And a single positive result is not where a screening programme stops. The U.S. Preventive Services Task Force issued its final recommendation statement on breast cancer screening on 30 April 2024: a B recommendation for biennial mammography in women aged 40 to 74. Its collaborative modelling priced the programme. “Collaborative modeling data estimated that a strategy of screening biennially from ages 40 to 74 years would result in 1376 false-positive results per 1000 women screened over a lifetime of screening”, alongside 16,116 mammograms, 8.2 breast cancer deaths averted and 14 overdiagnosed cases. A low base rate does not merely dilute one positive result. Repeated across a lifetime it produces more false positives than there are women.

Which quantity is considered variable decides what the numerical expression is saying. The sampling distribution p(y|θ) is a probability model over possible data for fixed θ. It integrates or sums to one over y, defines the observation mechanism, drives simulation and predictive checking, and conditions on parameter and model assumptions. The likelihood L(θ;y) is the same function viewed across θ for fixed observed y. It need not integrate to one over θ. It supports relative parameter comparison, it depends on the observed dataset, and it becomes a posterior only after combining with a prior and normalizing. The posterior p(θ|y) is a probability distribution over θ after observing y. It normalizes over parameter space, combines prior and likelihood, supports uncertainty intervals and predictions, and still depends on model adequacy.

The error the gynecologists and the physicians made is exactly this substitution. Sensitivity is a number attached to the observation mechanism, read off the p(y|θ) column. The question asked for a number in the p(θ|y) column. Reporting one where the other was requested is not a slip of arithmetic. It is a confusion about which quantity was being held fixed.

A prior encodes a scale, range, sparsity pattern, symmetry, or other belief before the current data. Even a broad prior can influence weakly identified directions. The phrase 'noninformative prior' is coordinate-dependent. A distribution flat in θ is not flat after a nonlinear transformation of θ. Sensitivity analysis should therefore compare plausible priors, especially when data are scarce, likelihoods are weak, or decisions depend on tails.

In at least one regulated setting that is not advice. It is an expectation with a docket number. The FDA issued its guidance on the use of Bayesian statistics in medical device clinical trials in February 2010, through CDRH and CBER. Section 5.7 is explicit: “Sensitivity analysis is used to investigate the effects of deviations from your statistical model and its assumptions. FDA may recommend that you submit a sensitivity analysis.” Among the deviations a sponsor may be asked to investigate, the section names “alternative prior distributions” and “alternative 'hyperprior' parameters in hierarchical models”. That is both the prior and the prior over the prior.

The same guidance supplies a unit for how much a prior is actually contributing. It defines an effective sample size, ESS = n × V1/V2, with V1 the variance without borrowing and V2 the variance with borrowing. The difference ESS − n is “the number of patients 'borrowed' from the previous trial”. That converts an argument about whether a prior is 'informative' into a countable quantity. A prior worth many borrowed patients is carrying a share of the conclusion, and a regulator can ask what happens when it is changed.

Posterior odds equal prior odds multiplied by a likelihood ratio. This form separates what was believed before the evidence from how strongly the evidence favors one hypothesis over another. Diagnostic tests, anomaly scores, and model comparison often become clearer in odds form. A large likelihood ratio can still leave posterior odds modest when the prior odds are very small.

The costliest demonstration of that arithmetic was a criminal trial. A jury was told that the chance of two cot deaths in such a family was “approximately a chance of 1 in 73 million”. That was Roy Meadow's figure, and he had obtained it by squaring 1 in 8,543 — that is, by treating the second death as independent of the first. The Royal Statistical Society said so publicly in a news release in October 2001: “The well-publicised figure of 1 in 73 million thus has no statistical basis.” The release also named the second error, the one that lives in the odds form: “This (mis-)interpretation is a serious error of logic known as the Prosecutor's Fallacy.” A small probability of the evidence under innocence is a likelihood, not posterior odds. It has to be set against how probable the same evidence is under the alternative, and the prior odds of that alternative are themselves very small.

On 11 April 2003 the Court of Appeal (Criminal Division) quashed Sally Clark's two murder convictions, [2003] EWCA Crim 1020. At paragraph 108 the judgment sets out the earlier Court of Appeal's holding on the figure: “Therefore we accept that when one is looking post facto at whether two deaths were natural or unnatural, the 1:73 million figure is no help. It is merely a distraction.” At paragraph 178 the court held for itself that “we think it very likely that it grossly overstates the chance of two sudden deaths within the same family from unexplained but natural causes.”

Base rates therefore remain essential even when evidence looks individually persuasive.

Treat prior choice, computation, and model checking as one process. Specify the generative story, writing how parameters produce latent variables and observations. Simulate from the prior, and check whether the implied datasets and parameter scales are plausible. Fit, and diagnose the computation. Check posterior predictions against important observed structures. Test sensitivity by varying priors, likelihoods, and influential observations.

The diagnosis step has published numbers rather than a feeling about convergence. The classic Gelman–Rubin R̂ diagnostic breaks in ordinary cases: “Traditional R̂ will fail to correctly diagnose convergence failures when the chain has a heavy tail or when the variance varies across the chains”. Vehtari and four co-authors showed that in Bayesian Analysis in 2021, and replaced the old 1.1 rule of thumb with two concrete thresholds. Use the posterior sample only if the rank-normalised R̂ < 1.01. Require a rank-normalised effective sample size > 400, computed over at least four chains. Of the tighter threshold they write: “This threshold is much tighter than the one recommended by Gelman and Rubin (1992), reflecting lessons learnt over more than 25 years of use, as well as the simulation results in Appendix A.” The Stan Development Team's Reference Manual adopts it by name: “Vehtari et al. (2021) suggest in general to use a threshold 1.01”.

Note how little of the workflow is about the algebra. One step is the model, one is the computation with its two published thresholds, and three are attempts to find out whether the thing is wrong.