next up previous
Next: Definition of Entropy and Up: Probability and entropy -- Previous: Probabilities and ensembles

Meaning of probability.

Probabilities can be used to describe frequencies of outcomes in random experiments, but giving a non-circular definition of a frequency is a challenge.

Probabilities can also be used, more generally, to describe degrees of belief  in propositions that do not involve random variables -- for example `the probability that 2050 will be the warmest year on record, assuming people don't change their lifestyle', or `the probability that the Hubble constant lies between 41 and 43, given measurements of the Sunyaev-Zel'dovich effect'. Degrees of belief can be mapped onto probabilities if they satisfy some simple consistency rules known as the Cox axioms . Thus probabilities can be used to describe assumptions, and to describe inferences given those assumptions. The rules of probability ensure that if two people make the same assumptions and receive the same data then they will draw identical conclusions. This more general use of probability is known as the Bayesian viewpoint. It is also known as the subjective  interpretation of probability, since the probabilities depend on assumptions. Advocates of a Bayesian approach to data modelling and pattern recognition do not view this subjectivity as a defect, since in their view, you can't do inference without making assumptions. In this book it will be taken for granted that a Bayesian approach makes sense, but the reader is warned that this is not yet a globally held view -- the field of statistics has been dominated for most of the 20th century by non-Bayesian methods in which probabilities are only allowed to describe random variables.



David J.C. MacKay
Sat May 10 23:05:10 BST 1997