CSC321 Winter 2009 - Lectures

• January 6
  Lecture 1: What are neural networks?
  (notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))
  Reading: How neural networks learn from experience ( .pdf)

• January 8
  Lecture 2: Two simple learning algorithms
  (notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))
  Reading: Connectionist Learning Procedures, pp 185-190; 193-198. ( .pdf)

• January 13
  Lecture 3: Learning in multilayer networks
  (notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))
  Reading: Connectionist Learning Procedures, pp 198-205.

• January 15
  Lecture 4: Learning to model relationships and word sequences
  (notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))
  Reading: A neural probabilistic language model. (.ps file)
  (a paper written for researchers about predicting the next word in a sentence).

• January 20: (Assignment 1 will be posted)
  Lecture 5: Applying backpropagation to shape recognition
  (notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))
  Reading(handout): Generalization and network design strategies
  Ignore section 2.2 and page 9.

• January 22
  Lecture 6: Learning in recurrent networks
  (notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))
  Reading: Learning internal representations by error propagation, pp 354-362. ( .pdf)

• January 27: Assignment 1 due (at start of class)
  Lecture 7: Overfitting and ways to fix it
  (notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))
  Reading: short web reading on generalization
  Reading: short web reading on overfitting
  Reading: short web reading on early stopping
  Reading: short web reading on weight decay
  

• January 29
  Lecture 8: The Bayesian way to fit models
  (notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))
  

• February 3: (Assignment 2 will be posted)
  Lecture 9: More on Bayesian model fitting
  (notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))
  

• February 5
  Lecture 10: Speeding up learning
  (notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))
  Reading: ()
  

• February 10: Assignment 2 due (at start of class)
  Lecture 11: Learning without a teacher: Autoencoders and PCA
  (notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))
  Reading: ()
  

• February 12
  Lecture 12: Clustering: The EM algorithm for fitting mixtures of Gaussians
  (notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))
  Reading: ()
  

• February 17 and 19 No Lectures (reading week)
  

• February 24: Midterm Test (1.10pm-2.00pm)
  

• Feb 26
  Lecture 13: Mixtures of experts
  (notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))
  Reading: Adaptive mixtures of local experts .pdf
  

• March 3: (Assignment 3 will be posted)
  Lecture 14: Hidden Markov Models
  (notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))
  Reading: Alan Poritz, Hidden Markov Models: A guided tour., ICASSP 1988. pdf

• March 5
  Lecture 15 (called 17): Learning Hidden Markov Models using EM
  (notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))
  Reading: Alan Poritz, Hidden Markov Models: A guided tour., ICASSP 1988. .pdf

• March 10: Assignment 3 due (at start of class)
  Lecture 16: Distributed representations
  (notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))
  Reading: Book chapter on "Distributed Representations" .pdf
  

• March 12
  Lecture 17: The effects of hardware damage on distributed representations
  (notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))
  Reading: Scientific American article on "Simulating Brain Damage" .pdf
  

• March 17: (Assignment 4 will be posted)
  Lecture 18: Hopfield Nets and simulated annealing
  (notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))
  Reading: For a gentle introduction to the idea of memories as energy minima ( .ps ) ( .html )
  Reading: For a gentle introduction to how to add new memories by creating new minima ( .ps ) ( .html )
  

• March 19
  Lecture 19: Boltzmann machines as probabilistic models
  (notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))
  

• March 24: Assignment 4 due (at start of class)
  Lecture 20: Learning in Boltzmann machines
  (notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))
  Reading: Scholarpedia entry on Boltzmann machines .pdf [ web page]
  

• March 26
  Lecture 21: Some demonstrations of learning in restricted Boltzmann machines
  (notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))
  

• March 31: (Assignment 5 will be posted)
  Lecture 22: Learning features one layer at a time
  (notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))
  Reading for lectures 22 and 23: "Learning multiple layers of representation". .pdf
  

• April 2
  Lecture 23: Sigmoid belief nets and the wake-sleep algorithm
  (notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page)) Reading for lectures 22 and 23: "Learning multiple layers of representation". .pdf
  

• April 7: Assignment 5 due (at start of class)
  Lecture 24: Using backpropagation to fine-tune deep networks
  (notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))
  Reading for lecture 24: "Reducing the dimensionality of data with neural networks" .pdf
  

• April 9
  Lecture 25: Learning distributed representations for sequential data. (notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page)) .pdf
  

• Friday April 24
  final exam (2 hours) 2.00 to 4.00 in East Hall, University College, 15 King's College Circle
  
  Map of Campus Buildings)
  

CSC321 - Computation In Neural Networks: || www.cs.toronto.edu/~hinton/csc321/