CSC321 Winter 2008 - Lectures

• January 8
  Lecture 1: What are neural networks?
  (notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))
  Reading(handout): How neural networks learn from experience.

• January 10
  Lecture 2: Two simple learning algorithms
  (notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))
  Reading(handout): Connectionist Learning Procedures, pp 185-190; 193-198.

• January 15
  Lecture 3: Learning in multilayer networks
  (notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))
  Reading(handout): Connectionist Learning Procedures, pp 198-205.

• January 17
  Lecture 4: Learning to model relationships and word sequences
  (notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))
  Reading: A neural probabilistic language model. (.ps file)
  (a paper written for researchers about predicting the next word in a sentence).

• January 22 : (Assignment 1 will be posted)
  Lecture 5: Applying backpropagation to shape recognition
  (notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))
  Reading(handout): Generalization and network design strategies
  Ignore section 2.2 and page 9.

• January 24
  Lecture 6: Learning in recurrent networks
  (notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))
  Reading(handout): Learning internal representations by error propagation, pp 354-362.

• January 29 : Assignment 1 due (at start of class)
  Lecture 7: Overfitting and ways to fix it
  (notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))
  Reading: short web reading on generalization
  Reading: short web reading on overfitting
  Reading: short web reading on early stopping
  Reading: short web reading on weight decay
  

• January 31
  Lecture 8: The Bayesian way to fit models
  (notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))
  

• February 5 (Assignment 2 will be posted: TA for this assignment is Andriy Mnih):
  Lecture 9: More on Bayesian model fitting
  (notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))
  

• February 7
  Lecture 10: Speeding up learning
  (notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))
  Reading: ()
  

• February 12 : Assignment 2 due (at start of class)
  Lecture 11: Learning without a teacher: Autoencoders and PCA
  (notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))
  Reading: ()
  

• February 14
  Lecture 12: Clustering: The EM algorithm for fitting mixtures of Gaussians
  (notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))
  Reading: ()
  

• February 19 and 21 No Lectures (reading week)
  

• February 26: Midterm Test (1.10pm-2.00pm)
  

• Feb 28
  Lecture 13: Mixtures of experts
  (notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))
  Reading(handout): Adaptive mixtures of local experts ( .pdf)
  

• March 4 (Assignment 3 will be posted)
  Lecture 14: Hidden Markov Models
  (notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))
  Reading: Alan Poritz, Hidden Markov Models: A guided tour., ICASSP 1988. [pdf7pages]

• March 6
  Lecture 15 (called 17): Learning Hidden Markov Models using EM
  (notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))
  Reading: Alan Poritz, Hidden Markov Models: A guided tour., ICASSP 1988. [ .pdf 7 pages]

• March 11 Assignment 3 due (at start of class)
  Lecture 16: Distributed representations
  (notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))
  Reading(handout): Book chapter on "Distributed Representations" to be handed out in class

• March 13 Assignment 4 will be posted
  Lecture 17: The effects of hardware damage on distributed representations
  (notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))
  Reading(handout): Scientific American article on "Simulating Brain Damage" to be handed out in class.

• March 18 (at start of class)
  Lecture 18: Hopfield Nets and simulated annealing
  (notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))
  Reading: For a gentle introduction to the idea of memories as energy minima ( .ps ) ( .html )
  Reading: For a gentle introduction to how to add new memories by creating new minima ( .ps ) ( .html )
  

• March 20 Assignment 4 due
  Lecture 19: Boltzmann machines as probabilistic models
  (notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))
  

• March 25
  Lecture 20: Learning in Boltzmann machines
  (notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))
  Reading: Scholarpedia entry on Boltzmann machines .pdf [ web page]
  

• March 27
  Lecture 21: Some demonstrations of learning in restricted Boltzmann machines
  (notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))
  

• April 1 (Assignment 5 will be posted)
  Lecture 22: Learning features one layer at a time
  (notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))
  Old Reading for lectures 22 and 23 (no longer required): "To recognize shapes, first learn to generate images" .pdf
  New Simplified Reading for lectures 22 and 23: "Learning multiple layers of representation". .pdf
  

• April 3
  Lecture 23: Sigmoid belief nets and the wake-sleep algorithm
  (notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))

• April 8 Assignment 5 due (at start of class)
  Lecture 24: Using backpropagation to fine-tune deep networks
  (notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))
  Reading for lecture 24: "Reducing the dimensionality of data with neural networks" .pdf
  

• April 10
  Lecture 25: Learning distributed representations for sequential data. (notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))

• April or May ??
  final exam (2 hours) Tuesday April 29 from 2.00 to 4.00 in BN3
  UofT building # ??
  Map of Campus Buildings)
  

CSC321 - Computation In Neural Networks: || www.cs.toronto.edu/~hinton/csc321/