Lectures

Here is a tentative schedule, which will likely change as the course goes on.

Suggested readings are just that: resources we recommend to help you understand the course material. They are not required, i.e. you are only responsible for the material covered in lecture.

ESL = The Elements of Statistical Learning, by Hastie, Tibshirani, and Friedman.
MacKay = Information Theory, Inference, and Learning Algorithms, by David MacKay.
Barber = Bayesian Reasoning and Machine Learning, by David Barber.
Bishop = Pattern Recognition and Machine Learning, by Chris Bishop.
Sutton and Barto = Reinforcement Learning: An Introduction, by Sutton and Barto.
Goodfellow = Deep Learning, by Goodfellow, Bengio, and Courville.

Week Topic(s) and Dates Slides & Suggested Readings Important Dates
Week 1 Introduction
Nearest Neighbours, 9/10
[Slides] [Video]

ESL: Chapters 1, 2.1-2.3, and 2.5
Metacademy: K nearest neighbors

Week 2 Decision Trees
Ensembles 9/17
[Slides] [Video]

ESL: 9.2, 2.9, 8.7, 15
Metacademy: decision trees, entropy, mutual information, bias/variance decomposition, bagging, random forests

9/13: Hw 1 out.
Week 3 Linear Regression
Linear Classifiers, 9/26
[Slides] [Video]

Bishop: 3.1, 4.1, 4.3
Course notes: linear regression, linear classifiers, logistic regression
Metacademy: linear regression, closed-form solution, gradient descent, ridge regression

Week 4 Softmax Regression
SVMs
Boosting, 10/1
[Slides] [Video]

Bishop: 7.1, 14.3
Course notes: optimization, SVMs and boosting

10/1: Hw1 due. 10/2: Hw2 out.
Week 5 PCA
K-Means
Maximum Likelihood, 10/8
[Slides] [Video] Bishop: 12.1, 9.1
Week 6 Probabilistic Models, 10/15 [Slides] [Video]

Bishop: 2.1-2.3, 4.2
MacKay: chapters 21, 23, 24
Course notes: probabilistic models

10/15: Hw2 due.
10/16: Hw3 out.
Week 7 Expectation-Maximization, 10/22 [Slides] [Video]

Bishop: 9.2-9.4
Barber: 20.1-20.3
Course notes: mixture models

10/22: midterm out.
10/25: midterm marks release.
Week 8 Neural Networks, 10/29 [Slides]

Bishop: 5.1-5.3
Course notes: multilayer perceptrons, backprop

Week 9 Convolutional Networks, 11/5 [Slides]

Course Notes: conv nets, image classification
Goodfellow, sections 9.1-9.5

11/5: Hw3 due. 11/6: Hw4 out.
Week 10 Reinforcement Learning, 11/12 [Slides]

Sutton and Barto: 3, 4.1, 4.4, 6.1-6.5

Week 11 Differential Privacy, 11/19 [Slides] Dwork and Roth, 2014. The Algorithmic Foundations of Differential Privacy. Chapters 2, 3.1-3.5. 11/20: Hw4 due. Final project out
Week 12 Algorithmic Fairness, 11/26 [Slides]

Barocas, Hardt, and Narayanan. Fairness and Machine Learning. Chapters 1 and 2.

Zemel et al., 2013. Learning fair representations.

Louizos et al., 2015. The variational fair autoencoder.

Hardt et al., 2016. Equality of opportunity in supervised learning.

Week 13 Work on final project
Week 14 Final project presentation, 12/10

Homeworks

Most homeworks will be due on Thursdays at 11:59pm. You will submit through MarkUs; directions are given in the assignment handouts.

just like old times
Out Due Materials TA Office Hours
Homework 1 9/13 10/1 [Handout]
Monday Sep 28th - 10-11 am (EST), Monday Sep 28th - 9-10 pm (EST)
Homework 2 10/2 10/15 [Handout]
[q1.py] [q2.py]
Tuesday Oct 13th - 10-10:30 am (EST), Tuesday Oct 13th - 9-9:30 pm (EST)
Homework 3 10/16 11/5 [Handout]
[code and data]
TBD
Homework 4 11/6 11/20 TBD TBD

Midterm

The midterm exam will be a take-home one. The exam will be distributed online, synchronously, and you have 24 hours to complete it offline. [Exam instructions] [Midterm.pdf]

Final project

The final project will replace what used to be a final exam and students are allowed to work in a team of two. There will be a final project presentation for evaluation during lecture time on week 14 Dec 10th. More details TBD

Website and course still under development, any feedback is very appreciated, please reach out to: Sheldon, email: huang at cs dot toronto dot edu