CIAR Summer School Tutorial

Lecture 2a

Products of Experts

How to combine simple density models

A picture of the two combination methods

Products of Experts and energies

How sharp are products of experts?

“Uni-gauss” experts

Combining energy dimples

Generating from a product of experts

Relationship to causal generative models

Learning a Product of Experts

Ways to deal with the intractable sum

The Markov chain for unigauss experts

A shortcut

A naļve model for binary data

A neural network for the naļve model

A mixture of naļve models

A neural network for a mixture of naļve models

A neural network for a product of naļve models

The distribution defined by one hidden unit

The logistic function computes a product of probabilities.

Restricted Boltzmann Machines

Restricted Boltzmann Machines and products of experts

A picture of the Boltzmann machine learning algorithm for an RBM

The short-cut

Contrastive divergence

Contrastive divergence