
CSC321 Winter 2013  LecturesTuesday January 8 First class meeting. Explaining how the course will be taught. For the rest of this schedule, students are to study the listed material before the class meeting. The 'lecture' meeting will always be a discussion of the material of that day. The 'tutorial' meetings are used for a variety of purposes. Thursday January 10 Tutorial: Introduction to Matlab, for novices (This is the beginning of a section: Introduction) Video 1a: Why do we need machine learning? Video 1b: What are neural networks? Video 1c: Some simple models of neurons Video 1d: A simple example of learning Tuesday January 15 Video 1e: Three types of learning Video 2a: Types of neural network architectures (This is the beginning of a section: Perceptrons) Video 2b: Perceptrons: The first generation of neural networks Thursday January 17 Tutorial: Introduction to Matlab, with focus on vectorizing Video 2c: A geometrical view of perceptrons Video 2d: Why the learning works Video 2e: What perceptrons can't do Tuesday January 22 (This is the beginning of a section: Modern NN basics) Video 3a: Learning the weights of a linear neuron Video 3b: The error surface for a linear neuron Video 3c: Learning the weights of a logistic output neuron Thursday January 24 Tutorial: The basics of probability theory and/or linear algebra Video 3d: The backpropagation algorithm Video 3e: Using the derivatives computed by backpropagation Tuesday January 29 (This is the beginning of a section: Language modeling) Video 4a: Learning to predict the next word Video 4b: A brief diversion into cognitive science Video 4c: Another diversion: The softmax output function Written lecture: The math of softmax units Video 4d: Neuroprobabilistic language models (Assignment 1 is posted: learning a distributed representation of words) Thursday January 31 Tutorial: Introducing assignment 1 Video 4e: Ways to deal with the large number of possible outputs (This is the beginning of a section: Architectures and why they work) Optionally revisit: Video 2a: Types of neural network architectures Video 101a: Coarse coding 1 Video 101b: Coarse coding 2 Video 101c: Coarse coding 3 Tuesday February 5 (Assignment 1 is due) Video 5a: Why object recognition is difficult Video 5b: Achieving viewpoint invariance Video 5c: Convolutional nets for digit recognition Video 5d: Convolutional nets for object recognition Thursday February 7 Tutorial: Post mortem on assignment 1 (This is the beginning of a section: Optimization) Video 6a: Overview of minibatch gradient descent Video 6b: A bag of tricks for minibatch gradient descent Tuesday February 12 Video 6c: The momentum method Video 6d: Adaptive learning rates for each connection Video 6e: Rmsprop: Divide the gradient by a running average of its recent magnitude (Assignment 2 is posted: optimization) Thursday February 14 Tutorial: Introducing assignment 2 (This is the beginning of a section: Recurrent neural networks) Video 7a: Modeling sequences: A brief overview Video 7b: Training RNNs with back propagation Tuesday February 19 and Thursday February 21, there are no class meetings (Reading week). Tuesday February 26 Video 7c: A toy example of training an RNN Video 7d: Why it is difficult to train an RNN Video 7e: Longterm Shorttermmemory Thursday February 28 Tutorial: midterm (This is the beginning of a section: Generalization) Video 9a: Overview of ways to improve generalization Video 9b: Limiting the size of the weights Video 9c: Using noise as a regularizer Tuesday March 5 (Assignment 2 is due) Video 9d: Introduction to the full Bayesian approach Video 9e: The Bayesian interpretation of weight decay Video 10a: Why it helps to combine models Video 10b: Mixtures of Experts Thursday March 7 Tutorial: Post mortem on assignment 2 Video 10c: The idea of full Bayesian learning Video 10d: Making full Bayesian learning practical Video 10e: Dropout Tuesday March 12 (This is the beginning of a section: Mixtures of Gaussians, as an introduction to generative models) Video 102a: Mixtures of Gaussians 1 Video 102b: Mixtures of Gaussians 2 Video 102c: Mixtures of Gaussians 3 Video 102d: Mixtures of Gaussians 4 (Assignment 3 is posted: Mixtures of Gaussians) Thursday March 14 Tutorial: Introducing assignment 3 (This is the beginning of a section: Undirected unsupervised stochastic models) Video 11a: Hopfield Nets Video 11b: Dealing with spurious minima Video 11c: Hopfield nets with hidden units Tuesday March 19 (Assignment 3 is due) Video 11d: Using stochastic units to improve search Video 11e: How a Boltzmann machine models data Video 12a: Boltzmann machine learning Video 12c: Restricted Boltmann Machines Thursday March 21 Tutorial: Post mortem on assignment 3 Video 12d: An example of RBM learning Video 12e: RBMs for collaborative filtering Video 14d: Modeling realvalued data with an RBM Tuesday March 26 (This is the beginning of a section: Directed unsupervised stochastic models) Video 13a: The ups and downs of back propagation Video 13b: Belief Nets Video 13c: Learning sigmoid belief nets Written lecture: The math of Sigmoid Belief Nets (Assignment 4 is posted: RBMs for pretraining) Thursday March 28 Tutorial: Introducing assignment 4 Video 13d: The wakesleep algorithm Video 14a: Learning layers of features by stacking RBMs Video 14b: Discriminative learning for DBNs Video 14c: What happens during discriminative finetuning? Tuesday April 2 (Assignment 4 is due) (This is the beginning of a section: Autoencoders) Video 15a: From PCA to autoencoders Video 15b: Deep autoencoders Video 15c: Deep autoencoders for document retrieval Thursday April 4 Tutorial: Post mortem on assignment 4 Video 15d: Semantic Hashing Video 15e: Learning binary codes for image retrieval Video 15f: Shallow autoencoders for pretrainingMap of Campus Buildings [ Home  Lectures, Readings, & Due Dates  Optional Readings  The Tutorials  Computing  Assignments  Tests  ] CSC321  Computation In Neural Networks:  www.cs.toronto.edu/~tijmen/csc321/ 