2
		Overview
	• We will combine two types of unsupervised neural net: – Undirected model  =  Boltzmann Machine – Directed model      =  Sigmoid Belief Net • Boltzmann Machine learning is made efficient by restricting the connectivity & using contrastive divergence. • Restricted Boltzmann Machines are shown to be equivalent to infinite Sigmoid Belief Nets with tied weights. – This equivalence suggests a novel way to learn deep directed belief nets one layer at a time. – This new method is fast and learns very good models, provided we do some fine-tuning afterwards • We can now learn a really good generative model of the joint distribution of handwritten digit images and their labels. – It is better at recognizing handwritten digits than discriminative methods like SVM’s or backpropagation.