CSC321 Spring 2003 - Lectures

Tentative Lecture Schedule: Subject to change

• January 7
  Lecture 1: What are neural networks?
  (notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))
  Reading: A general framework for parallel distributed processing.

• January 9
  Lecture 2: Learning with linear neurons
  (notes in postscript)
  Reading: Connectionist Learning Procedures, pp 185-190.

• January 14
  Lecture 3: Perceptrons
  (notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))
  Reading: Connectionist Learning Procedures, pp 195-198.

• January 16
  Lecture 4: Learning in multilayer networks
  (notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))
  Reading: Learning internal representations by error propagation, pp 318-352.

• January 21
  Lecture 5: Learning about relationships and phonemes
  No notes: The readings for this lecture and the next one contain everything you need.
  Reading: Connectionist Learning Procedures, pp 198-205

• January 23
  Lecture 6: Learning to deal with invariances
  (notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))
  Reading: Generalization and network design strategies (Le Cun) (hardcopy handed out in class)
  

• January 28: Assignment 1 due (at start of class)
  Lecture 7: Overfitting and ways to fix it
  (notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))
  Reading: short web reading on generalization
  Reading: short web reading on overfitting
  Reading: short web reading on early stopping
  Reading: short web reading on weight decay
  

• January 30
  Lecture 8: More on the Bayesian way to fit models
  This lecture covered the slides at the end of lecture 7.

• February 4
  Speeding up learning
  (notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))
  Reading: ()
  

• February 6
  Lecture 10: Learning in recurrent networks
  (notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))
  Reading: Learning internal representations by error propagation, pp 354-362.

• February 11: Assignment 2 due (at start of class)
  Lecture 11: Learning ensembles of networks
  (notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))
  Reading: ()
  

• February 13
  Lecture 12: Learning without a teacher
  (notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))
  Reading: ()
  

• February 18 and 20 No Lectures (reading week)
  

• February 25: Midterm Test (1.10pm-2.00pm)
  Reading: ()
  

• February 27
  Lecture 13: Clustering: The K-means algorithm
  (notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))
  Reading: ()
  

• March 4
  Lecture 14: The EM algorithm for mixtures of Gaussians
  (notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))
  Reading: ()
  

• March 6
  Lecture 15: Mixtures of experts
  Hardcopy of lecture notes handed out in class.
  Reading: Adaptive mixtures of local experts ( .ps file with no figures)
  

• March 11: Lecture 16: Hidden markov models
  (this is called lecture 15 on the slides. Ignore that!)
  (notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))
  Reading: Alan Poritz, Hidden Markov Models: A guided tour., ICASSP 1988. [pdf7pages]
  

• March 13: Assignment 3 due (at start of class)
  Lecture 17: Learning hidden markov models using EM
  (notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))
  (notes in Poritz's notation as .ppt ) (notes in Poritz's notation for all browsers)) (notes in Poritz's notation as .ps, 4 per page))
  Reading: Alan Poritz, Hidden Markov Models: A guided tour., ICASSP 1988. [pdf7pages]
  

• March 18:
  Lecture 18: Distributed representations
  (notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))
  Reading: Book chapter on "Distributed Representations" to be handed out in class

• March 20
  Lecture 19: The effects of hardware damage on distributed representations
  (notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))
  Reading: Scientific American article on "Simulating Brain Damage" to be handed out in class.

• March 25:
  Lecture 20: Hopfield Nets
  (notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))
  Reading: ()
  

• March 27: Assignment 4 due (at start of class)
  Lecture 21: Markov Random Fields and Gibbs sampling
  (notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))
  Reading: ()
  

• April 1:
  Lecture 22: Boltzmann machines
  (notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))
  

• April 3:
  Lecture 23: Learning in Boltzmann machines
  (notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))
  Reading: Paper on learning and relearning in Boltzmann machines (to be handed out)

• April 8:
  Lecture 24: Products of Experts
  (notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))
  Reading: "Products of Experts" (to be handed out)

• April 10: Assignment 5 due (at start of class)
  Lecture 25: Restricted Boltzmann Machines are Products of Experts
  (notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))
  Reading: ()
  

• April 21, 2.00-4.00pm final exam
  

CSC321 - Computation In Neural Networks: || www.cs.toronto.edu/~hinton/csc321/