CSC321: Neural Networks
 
Lecture 22
Learning features one layer at a time

Learning multilayer networks

Learning multi-layer networks (continued)

Recursive Restricted Boltzmann Machines

Recursive Restricted Boltzmann Machines

The overall model produced by composing two RBM’s

The generative model

Why does stacking RBM’s produce this kind of generative model?

How an RBM defines the probabilities of hidden and visible vectors

Why does layer-by-layer learning work?

An analogy

A guarantee

Back-fitting

A neural network model of digit recognition

See the movie at
http://www.cs.toronto.edu/~hinton/adi/index.htm

Samples generated by running the top-level RBM with one label clamped. There are 1000 iterations of alternating Gibbs sampling between samples.

Examples of correctly recognized MNIST test digits (the 49 closest calls)

How well does it discriminate on MNIST test set with no extra information about geometric distortions?

Slide 19

The receptive fields of the first hidden layer

The generative fields of the first hidden layer