Prior probability

Lore & Background

A prior can be determined from past information, such as previous experiments, or elicited from the subjective assessment of an experienced expert. When no information is available, an uninformative prior may be adopted as justified by the principle of indifference. In modern applications, priors are often chosen for mechanical properties like regularization and feature selection.

Reader's Guide

The concept of prior probability is central to Bayesian statistics, providing a formal mechanism to incorporate existing knowledge into statistical analysis. Its significance lies in enabling the updating of beliefs via Bayes' rule, producing posterior distributions that combine prior information with new data. The historical constraint of conjugate priors for tractability has been largely overcome by computational methods like Markov chain Monte Carlo, broadening the applicability of Bayesian methods. The construction of priors ranges from informative (based on past data or expert opinion) to weakly informative (for regularization) to uninformative (based on principles like indifference or invariance). The philosophical debate between objective and subjective Bayesianism highlights the tension between seeking logically required priors and acknowledging the subjective nature of prior beliefs. This debate, exemplified by arguments from Edwin T. Jaynes and others, underscores that priors represent a state of knowledge rather than an observer-independent feature of the world. The legacy of prior probability is its foundational role in Bayesian inference, enabling principled uncertainty quantification across scientific disciplines.

Did You Know?

• A prior can be determined from past information, such as previous experiments.
• The Haldane prior is an improper prior distribution with infinite mass.
• A strong prior is a type of informative prior where the prior dominates the data.
• The principle of indifference assigns equal probabilities to all possibilities.