CSC321 Home Lectures, Readings and Due Dates Optional Readings Tutorials Computing Assignments Tests Anthony Bonner's Homepage: www.cs.toronto.edu/~bonner

CSC321 Spring 2014 – Lectures, Readings and Due Dates The lectures are 11am - 1pm on Fridays in CC 2130. Tentative Schedule: January 10: Lecture 1: What are neural networks? (notes as .ppt) (notes as .pdf)) Reading: How neural networks learn from experience (.pdf) January 10: Lecture 2: Two simple learning algorithms (notes as .ppt) (notes as .pdf)) Reading: Connectionist Learning Procedures, pp 185-190; 193-198. (.pdf) January 17: Lecture 3: Learning in multilayer networks (notes as .ppt) (notes as .pdf) Reading: Connectionist Learning Procedures, pp 198-205. (.pdf) January 17: Lecture 4: Learning to model relationships and word sequences (notes as .ppt) (notes as .pdf) Reading: A neural probabilistic language model. (.pdf) (a paper written for researchers about predicting the next word in a sentence). January 24: Lecture 5: Distributed representations (notes as .ppt) (notes as .pdf) Reading: Book chapter on "Distributed Representations" (.pdf) Reading: Scientific American article on "Simulating Brain Damage" (.pdf) January 24: Lecture 6: Applying backpropagation to shape recognition (notes as .ppt) (notes as .pdf) Required viewing: http://yann.lecun.com January 28: Assignment 1 is posted. January 31: Lecture 7: Learning in recurrent networks (notes as .ppt) (notes as .pdf) Reading: Learning internal representations by error propagation, pp 354-362. (.pdf) January 31: Lecture 8: Modeling text using a recurrent neural network trained with a really fancy optimizer (notes as .ppt) (notes as .pdf) Reading: "Generating text with recurrent neural networks" (.pdf) February 4: Assignment 1 is due. February 7: Lecture 9: Ways of speeding up the learning and preventing overfitting (notes as .ppt) (notes as .pdf) Reading: short web reading on generalization Reading: short web reading on overfitting Reading: short web reading on early stopping Reading: short web reading on weight decay February 7: Lecture 10: The Bayesian way to fit models (notes as .ppt) (notes as .pdf) February 11: Assignment 2 is posted. February 14: Lecture 11: More on Bayesian model fitting (notes as .ppt) (notes as .pdf) February 14: Lecture 12: Combining models (notes as .ppt) (notes as .pdf) (notes on boosting as .pdf) February 21: No Lecture (reading week) February 25: Assignment 2 is due. February 28: Lecture 13: Learning without a teacher: Autoencoders and PCA (notes as .ppt) (notes as .pdf) Reading: () February 28: Lecture 14: Clustering: The EM algorithm for fitting mixtures of Gaussians (notes as .ppt) (notes as .pdf) Reading: () March 7: Midterm test, 11:10am - 12:00pm, in class. March 7: Lecture 15: Mixtures of experts (This material will not be covered and will not be on the exam) (notes as .ppt) (notes as .pdf) Reading: Adaptive mixtures of local experts (.pdf) March 7: Lecture 16: Hopfield Nets and simulated annealing (notes as .ppt) (notes as .pdf) Reading: For a gentle introduction to the idea of memories as energy minima (.pdf) (.html) Reading: For a gentle introduction to how to add new memories by creating new minima (.pdf) (.html) March 11: Assignment 3 is posted. March 14: Lecture 17: Boltzmann machines as probabilistic models (notes as .ppt) (notes as .pdf) March 14: Lecture 18: Learning in Boltzmann machines (notes as .ppt) (notes as .pdf) Reading: Scholarpedia entry on Boltzmann machines (.pdf) (web page) March 18: Assignment 3 is due. March 21: Lecture 19: Learning Restricted Boltzmann Machines (notes as .ppt) (notes as .pdf) March 21: Lecture 20: Learning features one layer at a time (notes as .ppt) (notes as .pdf) Reading for lectures 19 and 20: Optional extra reading for lectures 19 and 20: "Learning multiple layers of representation" (.pdf) March 25: Assignment 4 is posted. March 28: Lecture 21: Using backpropagation to fine-tune deep networks (notes as .ppt) (notes as .pdf) Reading for lecture 21: "Reducing the dimensionality of data with neural networks" (.pdf) March 28: Lecture 22: Transforming Autoencoders for learning the right representation of shapes. (notes as .ppt) (notes as .pdf) Reading for lecture 22: "Transforming Autoencoders" (.pdf) April 4: Lecture 23: Support Vector Machines: Part 1 (notes as .ppt) (notes as .pdf) April 4: Lecture 24: Support Vector Machines: Part 2 (notes as .ppt ) (notes as .pdf ) April 4: Assignment 4 is due.