CIAR 2005 Summer School Lectures

Lecture 1a: Mixtures of Gaussians, EM, and Variational Free Energy
(notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))
Reading: Neal, R.M. and Hinton, G.E. (1998)
A view of the EM algorithm that justifies incremental, sparse, and other variants
In: Learning in Graphical Models, M.I. Jordan (editor)
[abstract] [ps] [pdf]

Lecture 1b: Sigmoid Belief Nets
(notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))
Reading: Neal, R. M. (1990)
Learning stochastic feedforward networks.
Technical Report CRG-TR-90-7, Dept. of Computer Science, University of Toronto
[abstract] [ps] [pdf]
Reading: Frey, B. J. and Hinton, G. E. (1996)
A simple algorithm that discovers efficient perceptual codes
In L. Harris and M. Jenkin (Eds)
Computational and Biological Mechanisms of Visual Coding,
Cambridge University press, New York.
[abstract] [ps] [pdf]

Lecture 2a: Products of Experts
(notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))
Reading: Hinton, G. E. (2002)
Training Products of Experts by Minimizing Contrastive Divergence.
Neural Computation, 14, pp 1771-1800.
[Technical report version abstract]
[Technical report version ps.gz] [Technical report version pdf]

Lecture 2b: Learning a Deep Belief Net
(notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))
Reading: Hinton, G. E., Osindero, S. and Teh, Y. (2005)
A fast learning algorithm for deep belief nets.
(submitted to Neural Computation)
[ps.gz] [pdf]