The Bayesian approach
Consider a very simple linear model that only has two
parameters:
It is possible to display the full posterior distribution over
the two-dimensional parameter space.
The likelihood term is a Gaussian, so if we use a
Gaussian prior the posterior will be Gaussian:
This is a conjugate prior. It means that the prior is just
like having already observed some data.