CSC321 Winter 2007

Lecture 21: Some Demonstrations of Restricted Boltzmann Machines

A simple learning module:
A Restricted Boltzmann Machine

Weights à Energies à Probabilities

How to learn a set of features that are good for reconstructing images of the digit 2

Slide 5

Slide 6

Slide 7

Slide 8

Slide 9

Slide 10

Slide 11

Slide 12

Slide 13

Slide 14

Slide 15

Slide 16

Slide 17

Slide 18

Slide 19

Slide 20

Slide 21

Slide 22

Slide 23

How well can we reconstruct the digit images from the binary feature activations?

Some features learned in the first hidden layer for all digits

And now for something a bit more realistic

 A network with local connectivity

Slide 28

Training a deep network

The generative model after learning 3 layers