
CSC2535 Spring 2013  Lectures
DRAFT LECTURE SCHEDULE: SUBJECT TO CHANGE
The final versions of the lecture notes will generally be posted on the webpage around the time of the
lecture.
 January 9
Lecture 1: Overview of Machine Learning and Graphical Models
notes as ppt ,
notes as .pdf
Reading: Bishop, Chapter 8: pages 359399
 January 16
Lecture 2a: Inference in Factor Graphs
notes as ppt ,
notes as .pdf
Lecture 2b: Variational Inference and the wakesleep algorithm
notes as ppt ,
notes as .pdf
Reading: Bishop, Chapter 8: pages 399418, 450455
Reading: "The wakesleep algorithm for unsupervised Neural Networks."
.pdf
Reading: "A view of the EM algorithm that justifies incremental, sparse, and other variants."
.pdf
 January 23 (first assignment posted on web)
Lecture 3a: The origin of variational Bayes
notes as ppt ,
notes as .pdf
Required reading: Training Products of Experts by Minimizing Contrastive Divergence.
[pdf]
Lecture 3b: Approximate Learning Methods for EnergyBased Models
notes as ppt ,
notes as .pdf
Reading: "Keeping Neural Networks Simple by Minimizing the Description Length of the Weights."
.pdf
Reading: "Training products of experts by minimizing contrastive divergence"
.pdf
Reading: "Contrastive Backpropagation"
.pdf
 January 30
Lecture 4: Restricted Boltzmann machines
notes as ppt ,
notes as .pdf
Required reading: Training Restricted Boltzmann Machines using Approximations to
the Likelihood Gradient.
[pdf]
Reading: Estimation of nonnormalized statistical models using score matching.
[pdf]
 February 6: First assignment due (at start of class)
Lecture 5: Deep Boltzmann machines
notes as .ppt
notes as .pdf
Reading: "An efficient learning procedure for deep Boltzmann machines"
.pdf
 February 13 (second assignment posted on web)
Lecture 6: Object Recognition in Deep Neural Nets
notes as ppt ,
notes as .pdf
notes as ppt ,
notes as .pdf
Reading for lecture 6a: ImageNet Classification with Deep Convolutional Neural Networks.
[pdf]
Reading for lecture 6b: Transforming autoencoders.
[pdf]
Optional reading: A Fast Learning Algorithm for Deep Belief Nets.
[pdf]
 February 20 No Lecture (reading week)
 Feb 27 Second assignment due (at start of class)
Lecture 7: Models of words and documents
notes as .pdf
Required reading: Latent Dirichlet allocation.
[html]
 March 6
Lecture 8a: Learning threeway interactions
notes as .ppt ,
notes as .pdf
Required reading: Learning to represent spatial transformations with
factored higherorder Boltzmann machines.
[pdf]
Required reading: Factored Conditional Restricted Boltzmann Machines for Modeling Motion Style.
[pdf]
Lecture 8b: Autoencoders for Image retrieval:
notes as .ppt
notes as .pdf
.....Required reading: [.pdf]
 March 13
Lecture 9: Collaborative filtering and Missing Data Problems
notes as .pdf
Required reading: Learning from incomplete data.
[pdf]
Recommended reading: Recommender Systems: Missing Data and Statistical Model Es
timation.
[pdf]
 March 20
Lecture 10: Recurrent neural networks
notes as .ppt ,
notes as .pdf
Required reading: [.pdf]
 March 27
Lecture11: Nonlinear Dimensionality Reduction
notes as .ppt ,
notes as .pdf
Required Reading: ISOMAP
[article on web]
Required Reading: Local Linear Embedding
[.pdf]
Required Reading: Stochastic Neighbor Embedding
[.ps]
Required Reading: Visualizing Data using tSNE
[.pdf]
Optional reading: A paper that relates SNE to Laplacian Eigenmaps [.pdf]
Optional reading: Dimension Reduction: A Guided Tour.
[.pdf
]
 April 3
Final Test
(1.10pm2.00pm in the same place as the lectures)
 Monday April 15: Project due before midnight as .pdf sent to hinton@cs.toronto.edu
