CSC321 Lecture 25:

More on deep autoencoders
&
Using stacked, conditional RBM’s for modeling sequences

Do the 30-D codes found by the autoencoder preserve the class structure of the data?

Slide 3

The fastest possible way to find similar documents

Finding binary codes for documents

Making address space semantic

Where did the search go?

How good is a shortlist found this way?

Time series models

Time series models

The conditional RBM model

Comparison with hidden Markov models

Generating from a learned model

Three applications

An early application (Sutskever)

Show Ilya Sutskever’s movies

A hierarchical version

An application to modeling
motion capture data

An RBM with real-valued visible units
(you don’t have to understand this slide!)

Modeling multiple types of motion

Show Graham Taylor’s movies

Statistical language modelling

An application to language modeling

Factoring the weight matrices

How to compute a predictive distribution across 17000 words.

Slide 26

Slide 27

Slide 28

Slide 29

Slide 30