Inferring the (mu,sigma) of a Gaussian distribution, or a mixture of Gaussians, using Gibbs Sampling

Overview

Requirements

Note to me: create tar using

To run the demonstration:

1. get the demonstration files (41K) by clicking here http / ftp
2. unpack thus:

          tar xvf gibbsdemo.tar

   This unpacks a load of files into a directory called gibbsdemo. [If there are any complaints about the clock time use tar xvfm gibbsdemo.tar]

3.        cd gibbsdemo

4. (optional) modify your X windows defaults so that the gnuplot colours and fonts come out right. (Execute this command on the machine running X windows.)

          xrdb -load Xdefaults.gnu
          

5.  octave
          

   (Works fine with octave-2.0 but not with octave-2.1.33, which seems to have plotting bugs.)
6. To run the demo:

          load 'DEMO'
          

   Select from a range of demonstrations. Options [1] single-gaussian inference; gibbs sampling, [2] single-gaussian inference; priors on, [3] single-gaussian inference; silly data N=13, and [4] two-gaussian mixture model all work OK.

Commentary

• After viewing one option, you may find it necessary to quit and restart the demonstration in order to view another option with the correct defaults.
• The mixture of two Gaussians display. For the first few iterations, the following objects are displayed:
  1. The data (13 points in blue at the bottom)

      [ -0.01 -0.27 0.1 0.31  0.706   1.07 1.37 1.16  1.2 1.25 1.3 1.33 1.65 ]
          

  2. Assignments of those points to the two clusters (shown by yellow and orange points)
  3. The current two Gaussians (yellow and orange curves)
  4. The current mixture of two Gaussians (purple curve)
  After a few iterations have elapsed a fourth curve is displayed in green, which is the Monte Carlo average of the purple curve, i.e., the approximation to the predictive density for the next point.
  It is fun to leave the two-gaussian demo running for a long time to watch it switch between the two modes of the posterior density in which the small clump of points are respectively classed yellow and orange.
• The mixture of three Gaussians demo tends to blow up after a few minutes; I guess this is because I haven't switched on the priors on mu and sigma so there is a division by zero error when one of the clusters receives no points.