This assignment is about fitting mixtures of axis-aligned Gaussians to
two-dimensional data. Download and extract the archive below into your
directory:

http://www.cs.toronto.edu/~nitish/csc321/mog.tar.gz

PART 1 (5 points) 

Run moginit to create training and validation datasets from 5 random
axis aligned Gaussians.  Then use the function mogem to fit various numbers of
axis aligned Gaussians to the training data.  Using performance on the validation
data, determine the optimal number of Gaussians to fit to the training
data. Present your results as a graph that plots both the negative log validation
density and the negative log training density as a function of the number of
Gaussians. Include a brief statement of what you think the graph
shows. Also include a brief statement about the effects of changing
the initial standard deviation used in mogem. Use a standard deviation of 0.1
for making the plots.

Please do not change the random seeds in moginit.m (this will produce
different data). You can do this later if you want to just play around
and get a feel for how the algorithm behaves.


PART 2 (2 points)

Change moginit.m to use only 10 cases per Gaussian, and 10 axis-aligned
gaussians to generate the data and repeat the
experiment above (without changing the random seeds). Present your
results as a graph and include a brief statement of what you think the
graph shows and why it differs from the graph in PART 1.

PART 3 (3 points)

Change mogem.m so that in addition to fitting the means and
axis-aligned variances, it also fits the mixing proportions.
Currently, mogem does not mention mixing proportions so it is
currently assuming that they are all equal (which makes them all
cancel out when computing the posterior probability of each Gaussian
for each datapoint.).  So the first thing to do is to include mixing
proportions when computing the posterior, but keep them fixed (and not
all equal). Once you have debugged this, try learning the mixing
proportions.  Make mogem print out the final mixing proportions and
hand in final mixing proportions that you get when fitting 4 Gaussians to
the data, when you start with mixing proportions [0.25, 0.25, 0.25, 0.25],
[0.3, 0.2, 0.2, 0.3], [0.2, 0.3, 0.3, 0.2] and [0.9, 0.05, 0.025, 0.025].
Use the same settings to generate the data as you did in PART 2.