Deep Autoencoders
(Hinton & Salakhutdinov, Science 2006)
Some receptive fields of the first hidden layer
An autoencoder for patches of real faces
Reconstructions of face patches from new people
64 hidden units in the first hidden layer
Another test of the learning algorithm
Reconstructions from 30-dimensional codes
Do the 30-D codes found by the autoencoder preserve the class structure of the data?
A final example: Document retrieval
How to compress the count vector
Using autoencoders to visualize documents
A really fast way to find similar documents
Conditional Boltzmann Machines (1985)
What can conditional Boltzmann machines do that backpropagation cannot do?
Conditional BM’s without hidden units
Higher order Boltzmann machines
Using higher order Boltzmann machines to model transformations between images.
Higher order conditional Boltzmann machines
Another picture of a
conditional,
higher-order Boltzmann machine
A three stage training
procedure
(Taylor, Hinton and Roweis)
Generating from a learned model
Comparison with hidden Markov models
An application to modeling
motion capture data