First copy the files below into your directory. You can get them from
the web or from CDF.

http://www.cs.toronto.edu/~hinton/csc321/matlab/classbp1.m
http://www.cs.toronto.edu/~hinton/csc321/matlab/showweights.m
http://www.cs.toronto.edu/~hinton/csc321/matlab/assign1data.mat
http://www.cs.toronto.edu/~hinton/csc321/matlab/show.m
http://www.cs.toronto.edu/~hinton/csc321/matlab/showimages.m

Then invoke matlab (preferably without the graphical user interface!).
This can be done by typing
matlab -nojvm

At the >> prompt, type 

load assign1data.mat

This will create matrices called data, testdata, targets, testtargets.
data consists of 150 images. There are 30 fives, 30 sixes, 30 sevens,
30 eights and 30 nines. Each image has 256 pixels and is a row of the
matrix. testdata is a test set with 600 examples of each class. The
reason for making the test set so much bigger is to allow you to see 
what happens during overfitting without too much noise from the quirks 
of the test set.

You can type
showimages(data)
to see images of the training data.

You can type
showimages(testdata(601:700,:))
to see images of the first 100 sixes in the test set.

Now type

restart=1;
maxepoch=1000;
epsilon=.005;
numhid=20;
classbp1;

classbp1 uses softmax output units and a cross-entropy error (the error
on a case is the negative log probability that the network gives to
the correct answer).

classbp1 prints out the cross-entropy (E) on the training set and on
the test set. It also prints out the number of errors if you
choose the maximum output as the right answer. When it has finished it
plots graphs of the number of test errors and of the cross-entropy
cost on the test set. The plots will be on top of each other and you
will need to move the top window to see the one underneath.

If you call showweights(inhid, hidout) you can also print out the
input-to-hidden weights as gray-level images.  The top row of each of
these images is the weights from that hidden unit to the 5 outputs,
so you can see what effect each hidden unit has.  Using a small number
of hidden units (e.g. 10), try to make sense of the weights by seeing
which classes each hidden feature votes for. classbp1 calls
showweights every 500 epochs, but you can change it if you want to
watch the weights change.

QUESTION 1 (5 points)

Try various settings of numhid (use all powers of 2 from 1 to 256) and
watch what happens to both types of error on both the training set and
the test set.  Write less than one page describing what you discover. 
You may include printouts or graphs in addition to the one page.
Do not hand in computer code.

QUESTION 2 (3 points) 

Using numhid=50, try changing the learning rate, epsilon, and observe
what happens as you try bigger and bigger values for epsilon.  Try
values of epsilon from 1/2048 to 1/2 making each value twice as big as
the previous one.  It may help to display the hidden activities for
all the training cases. You can do this by typing: 
figure(2); clf; show(hidacts'); 
Write less than half of one page describing what you discover. You may
include printouts or graphs in addition to the half page.
Do not hand in computer code.