CSC2535 Spring 2006 - Lectures

LECTURE SCHEDULE: SUBJECT TO CHANGE

The real lecture notes will be posted around the time of the lecture. The notes for future lectures are from a previous course and will change. But they give a rough guide to the level and the content of the 2006 course.

January 11
Lecture 1: The history of neural networks
notes as .htm , notes as .ppt , notes as .ps, 4 per page
Required reading (handout): Connectionist Learning Procedures
This is a review paper that covers lots of topics. Different bits of it will be relevant to different lectures. For the first 3 lectures pages 185-209 are relevant.
January 18
Lecture 2: Some examples of backpropagation learning
notes as .htm , notes as .ppt , notes as .ps, 4 per page
Required reading: Paper on the language model .ps file)
Required reading: Paper on the LeNet5 character recognizer lecun-98.ps.gz pages 1-30
Required browsing: Try all Yann Le Cun's demos
Optional reading: Paper on the model that learns to draw the digits .ps file)
January 25
Lecture 3: Making backpropagation generalize better and using backpropagation for unsupervised learning
notes as .htm , notes as .ppt , notes as .ps, 4 per page
Required reading: Chapter 7 of Bishop's new book (handed out in class) except for pages 331-337; 346-349; 355-end.
Required reading: Becker and Hinton 1992
Optional reading: Slow feature analysis discovers a rich repertoire of complex cell properties. [.pdf ]
Optional reading: Learning a similarity metric discriminatively with application to face verification. [.pdf ]
February 1 (first assignment posted on web)
Lecture 4: Boltzmann Machines, Sigmoid Belief Nets and Gibbs Sampling
notes as .htm , notes as .ppt , notes as .ps, 4 per page
notes on Markov Chain Monte Carlo as .pdf
Required reading: Connectionist Learning Procedures pp 209-212.
Required reading: Learning stochastic feedforward networks. [ps.Z] [pdf]
February 8: First assignment due (at start of class)
Lecture 5: Variational learning
notes as .htm , notes as .ppt , notes as .ps, 4 per page
Required Reading: Neal and Hinton (1997) [ps.gz] [pdf]
Optional reading: Frey, B. J. and Hinton, G. E. (1996) [ps] [pdf]
Optional Reading: Chapter 33 of MacKay's book .ps.gz or .pdf
February 15
Lecture 6a: Improving generalization by making the weights cheap to describe
notes as .htm , notes as .ppt , notes as .ps, 4 per page
Required reading: Hinton, G. E. and van Camp, D. (1993) [ps]
Optional reading: Simplifying neural networks by soft weight-sharing
[]
Lecture 6b: Variational Bayes notes as .pdf , notes as .ps
February 22 No Lecture (reading week)
March 1: Midterm Test (12.10am-1.10pm)
Lecture 7: (1.15-2.00pm) Independent Components analysis
notes as .htm , notes as .ppt , notes as .ps, 4 per page
Reading: Chapter 34 of David Mackay's book [ps] [pdf]
March 8 (second assignment posted on web)
Lecture 8: Products of Experts and Contrastive Divergence
notes as .htm , notes as .ppt , notes as .ps, 4 per page
Reading: Training Products of Experts by Minimizing Contrastive Divergence
[ps.gz] [pdf]
March 15: Second assignment due (at start of class)
Lecture 9: Learning multiple layers of features greedily.
notes as .htm , notes as .ppt , notes as .ps, 4 per page
Lecture 9 (extra material)
notes as .htm , notes as .ppt , notes as .ps, 4 per page
Reading: A fast learning algorithm for deep belief nets. [ps.gz] [pdf]
March 22
Lecture 10: Deterministic Energy-Based Models
notes as .htm , notes as .ppt , notes as .ps, 4 per page
Reading: Contrastive backpropagation [ps.gz] [pdf]
Reading: Energy-based models for sparse overcomplete representations. [ps.gz] [pdf]
March 29
Lecture 11: Conditional Random Fields
notes as .htm , notes as .ppt , notes as .ps, 4 per page
Reading: Introduction to Conditional Random Fields [pdf]
April 5
Lecture 12: Non-linear dimensionality reduction
notes as .htm , notes as .ppt , notes as .ps, 4 per page
Reading: A Global Geometric Framework for Nonlinear Dimensionality Reduction
article on web
Reading: Stochastic Neighbor Embedding [.ps]
April 12: Final Test (12.10am-1.10pm)
Lecture 13: Representing things with neurons (1.15-2.00)
LAST YEAR'S NOTES: notes as .htm , notes as .ppt , notes as .ps, 4 per page
Reading: Mapping Part-Whole hierarchies into connectionist networks (to be handed out)
Reading: Holographic reduced representations: Convolution Algebra for Compositional Distributed Representations (to be handed out)
Tues April 18: Project due at Pratt 290G by 5.00pm

CSC2535 - Computation In Neural Networks: || www.cs.toronto.edu/~hinton/csc2535/