Calendar

There are two sections for the course.

• For the afternoon section, the first lecture each week is 1:10-2pm on Tuesday, and the second lecture is 1:10-2pm on Thursday. The tutorial is 12:10-1pm on Thursday. Lectures and tutorial are both in BA1200.
• For the night section, the two lectures are held back-to-back from 6:10-7:50pm. The tutorial is 8:20pm-9pm. Lectures and tutorial are both in BA1220.

Each week has some assigned Coursera videos. The quiz on those videos is due by 11:59pm on Monday of that week. Since the in-class meetings build on the material in the Coursera videos, it is important that you watch them before class.

No lectures, tutorials, or office hours will be held for the duration of the CUPE 3902 Unit 1 strike. You should have received an e-mail with further details.

Week 1, Jan 5-9: Introduction

• Background: If it’s been a long time since you used any linear algebra, this is a good time for a refresher. Here are the relevant concepts on Metacademy. Each one gives a number of pointers, but the Khan Academy links are especially useful since they have auto-graded exercises you can use to check your understanding.
  • vectors
  • dot product
  • linear systems as matrices
  • matrix multiplication
  • matrix inverse
  • matrix transpose
• Lecture 1: What are neural networks and machine learning? (Traditional lecture) pdf
• Lecture 2: Linear regression (Traditional lecture) pdf
• No tutorial this week.
• Optional resources:
  • linear regression
  • basis function expansions

Week 2, Jan 12-16: Perceptron learning

• Assigned Coursera videos: Lectures A and B. Remember that both quizzes are due at 11:59pm on Monday, Jan. 12. You may find these lecture notes a helpful supplement.
• Background: plotting linear inequalities in 2 dimensions
• Lecture 3: Binary linear classification [pdf]
• Lecture 4: The perceptron learning algorithm [pdf] (We didn’t make it through all of these slides in lecture, so we will finish the rest next week.)
• Tutorial: Python programming [pdf]
  • This covers the basics of the Python language, NumPy arrays, and plotting with Matplotlib.
• Optional resources:
  • binary linear classifiers
  • perceptron algorithm

Week 3, Jan 19-23: Backpropagation

• Assigned Coursera videos: Lecture C. The quiz is due 11:59pm Monday as usual. [notes]
• Background:
  • We compute a lot of derivatives this week. If you want a refresher, try the Khan Academy sequence on computing derivatives, which includes some auto-graded exercises. The sections on the chain rule and the product and quotient rules are especially relevant.
  • We will also work with partial derivatives.
  • This week is when we start to talk about implementing learning algorithms. See the Homework page for some recommended reading on Python and NumPy, as well as installation instructions.
• Lecture 5: Learning in a single neuron [pdf]
• Lecture 6: Backpropagation [pdf]
• Tutorial: Implementing the learning in Python ([pdf] [code])
• Optional resources:
  • feed-forward neural nets
  • backpropagation
  • gradient descent

Week 4, Jan 26-30: Neural language models and optimization

• Assigned Coursera videos: Lectures D and E. Note that we are going a bit out of order compared with how the Coursera course was originally taught. [notes]
• Background:
  • probability
  • independent events
  • conditional probability
  • Bayes’ Rule
  • expectation and variance
• Lecture 7: Neural language models [pdf]
• Lecture 8: Optimization tricks [pdf]
• Tutorial: Introducing Assignment 1 [pdf]
• Optional resources:
  • notes on the math for softmax units
  • the original paper on neural language models
  • stochastic gradient descent
• Extra office hours for Assignment 1: Friday from 3-4pm in BA4161

Week 5, Feb 2-6: Recurrent neural nets

• Assigned Coursera videos: Lecture F [notes]
• Assignment 1 is due at the start of lecture on Tuesday.
• Lecture 9: Recurrent nets [pdf]
• Lecture 10: Training recurrent nets [pdf]
• No tutorial this week.

Week 6, Feb 9-13: Convolutional nets

• Assigned Coursera videos: Lecture G [notes]
• Lecture 11: Convolutional nets [pdf]
• Lecture 12: Recent advances in convolutional nets (traditional lecture) [pdf]
• Tutorial: Assignment 1 post-mortem; introducing Assignment 2
• Optional resources:
  • convolutional nets

Reading week, Feb 16-20

There is no class during reading week, but we will still hold office hours. There are no office hours on Monday, since it is a holiday. Instead, we’ll hold them at the following times:

• Tuesday, Feb 17, from 2-4pm in Pratt 290C (the usual room)
• Friday, Feb 20, from 3-5pm in Pratt 290C

Week 7, Feb 23-27: Midterm exam, and Generalization

This week, we will have an unusual schedule because of the midterm. See the announcements page for more information about the test.

• For the afternoon section:
  • The midterm will be held during lecture, 1:10-2pm on Tuesday, Feb. 24, in the usual room.
  • In place of the tutorial, there will be a double-lecture on generalization from 12:10-2pm on Thursday, Feb. 26.
• For the night section:
  • The midterm will be held during lecture, 6:10-7pm on Tuesday, Feb. 24, in the usual room.
  • In place of the tutorial, there will be a double-lecture on generalization from 7:10-9pm on Tuesday, Feb. 24.

This week’s content:

• Assigned Coursera videos: Lectures J and K
  • You only have to do the quiz for Lecture J.
  • You are still responsible for the first video in Lecture K, even though there is no quiz. The other videos for Lecture K are optional.
  • Because of the midterm, we have extended the quiz deadline until 11:59pm Thursday. However, you will get more out of class if you watch the videos beforehand.
• Background:
  • conditional probability
  • Bayes’ Rule
  • expectation and variance
  • Gaussian distribution
• Lecture 13 + 14: Improving generalization [pdf] (solutions will be posted after we finish covering the material)
• Tutorial: introducing assignment 2 [slides]
• Midterm solutions:
  • afternoon (we discussed these during tutorial)
  • night (we will discuss these at the start of class next week)
• Optional resources:
  • generalization
  • cross-validation
  • regularized linear regression
  • LASSO (sparse linear regression)
  • bagging

Week 8, March 2-6: Probabilistic models

No lectures, tutorials, or office hours will be held for the duration of the CUPE 3902 Unit 1 strike. You should have received an e-mail with further details.

• Assigned Readings: instead of videos, we have required readings for this week. Even though there’s no quiz, you should do the readings before lecture. Don’t be intimidated by the length: most of the text is worked-through examples. There are fewer new ideas than in most of the video lectures.
  • learning probabilistic models
  • mixture models
• Background: look at the Week 7 background if you haven’t done so yet.
• Lecture 15: Learning probabilistic models
• Lecture 16: Mixture models
• We no longer have tutorials, since both sections voted to replace the tutorial with another hour of lecture.
• Optional resources:
  • maximum likelihood
  • Bayesian parameter estimation
  • MAP estimation
  • mixture of Gaussians models
  • Expectation-Maximization

Week 9, March 9-13: Boltzmann machines

• Assigned Coursera videos: Lectures L and M
  • Note that you have two quizzes. Due to the assignment deadline, the quiz deadline is moved to 11:59pm on Thursday, March 12.
• Assignment 2 is due at the start of lecture on Tuesday.
• Lecture 17: Hopfield nets and Boltzmann machines
• Lecture 18: Learning Boltzmann machines

Week 10, March 16-20: Autoencoders

• Assigned Coursera videos: Lecture N
• Assignment 3 is due at the start of lecture on Tuesday.
• Lecture 19: Principal component analysis
• Lecture 20: Autoencoders

Week 11, March 23-27: Bayesian neural networks

• We will have assigned readings rather than videos.
• Lecture 21: Bayesian neural networks
• Lecture 22: Bayesian optimization
• Tutorial: Assignment 3 post-mortem; introducing Assignment 4

Week 12, March 30 to April 3: Reinforcement learning

• We will have assigned readings rather than videos.
• Assignment 4 is due at the start of lecture on Tuesday.
• Lecture 23: Q-learning
• Lecture 24: Policy gradient
• Tutorial: final exam review