CIAR Summer School Tutorial

Lecture 2a

Products of Experts

How to combine simple density models

A picture of the two combination methods

Products of Experts and energies

How sharp are products of experts?

Uni-gauss experts

Combining energy dimples

Generating from a product of experts

Relationship to causal generative models

Learning a Product of Experts

Ways to deal with the intractable sum

The Markov chain for unigauss experts

A shortcut

A nave model for binary data

A neural network for the nave model

A mixture of nave models

A neural network for a mixture of nave models

A neural network for a product of nave models

The distribution defined by one hidden unit

The logistic function computes a product of probabilities.

Restricted Boltzmann Machines

Restricted Boltzmann Machines and products of experts

A picture of the Boltzmann machine learning algorithm for an RBM

The short-cut

Contrastive divergence

Contrastive divergence