Learning Energy-Based Models of
High-Dimensional Data

Discovering causal structure as a goal for unsupervised learning

A different kind of hidden structure

Two types of density model

Bayes Nets

Approximate inference

A trade-off between how well the model fits the data and the tractability of inference

Energy-Based Models with deterministic hidden units

Maximum likelihood learning is hard

Hybrid Monte Carlo

Slide 11

Backpropagation can compute the gradient that Hybrid Monte Carlo needs

The online HMC learning procedure

A surprising shortcut

Intuitive motivation

Contrastive divergence

Contrastive divergence

Frequently Approximately Satisfied constraints

Learning constraints from natural images
(Yee-Whye Teh)

Slide 20

Slide 21

How to learn a topographic map

Slide 23

Faster mixing chains

Slide 25

Slide 26

Slide 27

Slide 28

Where to find out more