CIAR Summer School Tutorial
Learning a Deep Belief Net
A neural network model of digit recognition
The generative model
Learning by dividing and conquering
Another way to divide and conquer
Why its hard to learn one layer at a time
Using complementary priors to eliminate explaining away
An example of a complementary prior
Inference in a DAG with replicated weights
A picture of the Boltzmann machine learning algorithm for an RBM
"The learning rule for a..."
Proís and conís of replicating the weights
†A quick way to learn an RBM
Multilayer contrastive divergence
A simplified version with all hidden layers the same size
Why the hidden configurations should be treated as data when learning the next layer of weights
Why greedy learning works
Samples generated by running the top-level RBM with one label clamped. There are 1000 iterations of alternating Gibbs sampling between samples.
Examples of correctly recognized MNIST test digits (the 49 closest calls)
How well does it discriminate on MNIST test set with no extra information about geometric distortions?
Samples generated by running top-level RBM with one label clamped. Initialized by an up-pass from a random binary image. 20 iterations between samples.
The wake-sleep algorithm
The flaws in the wake-sleep algorithm
The up-down algorithm:
A contrastive divergence version of wake-sleep
The receptive fields of the first hidden layer
The generative fields of the first hidden layer
A different way to capture low-dimensional manifolds