Machine learning is a set of techniques that allow machines to learn from data and experience, rather than requiring humans to specify the desired behavior by hand. Over the past two decades, machine learning techniques have become increasingly central both in AI as an academic field, and in the technology industry. This course provides a broad introduction to some of the most commonly used ML algorithms.
The first half of the course focuses on supervised learning. We begin with nearest neighbours, decision trees, and ensembles. Then we introduce parametric models, including linear regression, logistic and softmax regression, and neural networks. We then move on to unsupervised learning, focusing in particular on probabilistic models, but also principal components analysis and Kmeans. Finally, we cover the basics of reinforcement learning.
There are two sections of the course. Tutorials will be held in the main lecture room.
Lecture Time  Tutorial Time  Lecture/Tutorial Room  Start  End  
Section 1  Wednesday 10amnoon  Wednesday noon1pm  Bahen 1190  Sept. 11  Nov. 27 
Section 2  Thursday 24pm  Thursday 45pm  Bahen 1180  Sept. 12  Nov. 28 
Prerequisites (an undergrad course in each is sufficient):
Marking Scheme.
Collaboration policy. You are expected to work on the homeworks by yourself. You should not discuss them with anyone except the TAs or the instructor.
Academic Integrity. By this point in your studies, you've heard this lots of times, so we'll keep it brief: avoid academic offenses (i.e. cheating). All graded work in this course is individual work.
Lateness. Homeworks will be accepted up to 3 days late, but 10% will be deducted for each day late, rounded up to the nearest day.
Remarks. Remark requests for homeworks should be made through MarkUs, and will be considered by the same TA who marked the assignment. The deadline for requesting a remark is typically one week after the marked assignments are returned. Remark requests for exams will be handled by the instructor; details to be announced later.
Exceptions. Exceptions to the course policies such as late homeworks or missed tests require permission of the instructor. For medical excuses, you should obtain an official Student Medical Certificate.
Auditing. If you are a U of T student, then you may audit the course (i.e. sit in on the lectures) only if there are empty seats available after everyone enrolled in the course has been seated. Anyone else (i.e. nonstudents) is not permitted to audit; this is a University policy. No University resources will be committed to the auditor, i.e. we won't mark homeworks or exams.
Most homeworks will be due on Thursdays at 11:59pm. You will submit through MarkUs; directions are given in the assignment handouts.
Out  Due  Materials  TA Office Hours  
Homework 1  9/13  9/26 
[Handout] [clean_real.txt] [clean_fake.txt] [clean_script.py] 
Fri 9/20, 121pm, in BA3201 Mon 9/23, 11amnoon, in BA3201 Wed 9/25, 24pm, in BA3201 Thu 9/26, 11amnoon, in BA3201 
Homework 2  9/25  10/10 
[Handout] [q2.py] 
Fri 10/4, 121pm, in BA3201 Mon 10/7, 11amnoon, in BA3201 Wed 10/9, 24pm, in BA3201 Thu 10/10, 11amnoon, in BA3201 
Homework 3  10/11 
[Handout] 
Fri 10/18, 121pm, in BA3201 Mon 10/21, 11amnoon, in BA3201 Wed 10/23, 24pm, in BA3289 Thu 10/24, 11amnoon, in BA3289 

Homework 4  11/1  11/14 
[Handout] [Code and Data] 
Fri 11/8, 121pm, in BA3201 Fri 11/8, 67pm, in BA3201 Mon 11/11, 11amnoon, in BA3201 Mon 11/11, 24pm, in BA3201 Thu 11/14, 11amnoon, in BA3201 
Homework 5  11/15  11/28 
[Handout] 
Wed 11/20, 23pm, in BA3201 Mon 11/25, 11amnoon, in BA3201 Wed 11/27, 34pm, in BA3201 Wed 11/27, 67pm, in BA3201 Thu 11/28, 11amnoon, in BA3201 
The course will have a midterm and a final exam.
The midterm will be held from 4:10pm to 5:40pm on Wednesday, Oct. 30, in the Health Sciences building, room 610. See Lecture 6 slides for more information. You might find the following practice exams helpful:
There will be office hours for midterm according to the following
schedule:
Fri 10/25, 121pm, in BA3201
Fri 10/25, 67pm, in BA3201
Mon 10/28, 11amnoon, in BA3201
Tue 10/29, 24pm, in BA3201
Wed 10/30, noon1pm, in BA1190 (lecture room)
Here are the midterm questions and solutions.
The final exam will be held from 3pm to 6pm on Tuesday, Dec. 17, in the Banting Institute, room 131.
There will be office hours for final exam according to the following
schedule:
Mon 12/2, 121pm, in BA5287
Tue 12/3, 121pm, in BA3289
Wed 12/4, 1112am, in BA3201
Thu 12/5, 57pm, in BA3201
Fri 12/6, 121pm, in BA3201
There will be no office hours the week before the exam (Dec 913) as TAs are
away attending the NeurIPS conference. Please ask your questions on Piazza.
Here is the Fall 2018 final, for practice. (This one was a bit on the hard side. Question 4 was not covered this year.)
Here is a tentative schedule, which will likely change as the course goes on.
Suggested readings are just that: resources we recommend to help you understand the course material. They are not required, i.e. you are only responsible for the material covered in lecture.
ESL = The Elements of Statistical Learning, by Hastie, Tibshirani, and Friedman.
MacKay = Information Theory, Inference, and Learning Algorithms, by David MacKay.
Barber = Bayesian Reasoning and Machine Learning, by David Barber.
Bishop = Pattern Recognition and Machine Learning, by Chris Bishop.
Sutton and Barto = Reinforcement Learning: An Introduction, by Sutton and Barto.
Goodfellow = Deep Learning, by Goodfellow, Bengio, and Courville.
Topic(s)  Dates  Slides  Suggested Readings  
Lecture 1  Introduction Nearest Neighbours 
9/11, 9/12  [Slides] 
ESL: Chapters 1, 2.12.3, and 2.5 
Lecture 2  Decision Trees Ensembles 
9/18, 9/19  [Slides] 
ESL: 9.2, 2.9, 8.7, 15 
Lecture 3  Linear Regression Linear Classifiers 
9/25, 9/26  [Slides] 
Bishop: 3.1, 4.1, 4.3 
Lecture 4  Softmax Regression SVMs Boosting 
10/2, 10/3  [Slides] 
Bishop: 7.1, 14.3 
Lecture 5  Neural Networks  10/9, 10/10  [Slides] 
Bishop: 5.15.3 
Lecture 6  Convolutional Networks  10/16, 10/17  [Slides] 
Course Notes: conv nets, image classification 
Lecture 7  PCA KMeans Maximum Likelihood 
10/23, 10/24  [Slides]  Bishop: 12.1, 9.1 
Lecture 8  Probabilistic Models  10/30, 10/31  [Slides]  Bishop: 2.12.3, 4.2 
Lecture 9  ExpectationMaximization  11/6, 11/7  [Slides]  Bishop: 9.29.4 
Lecture 10  Reinforcement Learning  11/13, 11/14  [Slides]  Sutton and Barto: 3, 4.1, 4.4, 6.16.5 
Lecture 11  Differential Privacy  11/20, 11/21  [Slides]  Dwork and Roth, 2014. The Algorithmic Foundations of Differential Privacy. Chapters 2, 3.13.5. 
Lecture 12  Algorithmic Fairness  11/27, 11/28  [Slides]  Barocas, Hardt, and Narayanan. Fairness and Machine Learning. Chapters 1 and 2. Zemel et al., 2013. Learning fair representations. Louizos et al., 2015. The variational fair autoencoder. Hardt et al., 2016. Equality of opportunity in supervised learning. 
Topic  Dates  Materials  
Tutorial 1  NumPy review, KNearestNeigbors  9/11, 9/12 
[ipynb] Reviews: [Linear algebra slides], [NumPy basics], [ipynb ex1], [SVD slides], [ipynb SVD], [ipynb ex2] 
Tutorial 2  eigendecompositions, SVD, basic information theory  9/18, 9/19  [Slides] 
Tutorial 3  Gradient Descent  9/25, 9/26 
[Slides] [Lecture ipynb] [Worksheet ipynb] [Convexity] 
Tutorial 4  Random Forests and XGBoost  10/2, 10/3  [Slides] 
Tutorial 5  Autograd + PyTorch  10/9, 10/10  [Autograd ipynb] [Pytorch ipynb] 
Tutorial 6  Convnets  10/16, 10/17  [CNNs ipynb] 
Tutorial 7  Midterm Review  10/23, 10/24  [Slides] 
  Midterm, No Tutorials  10/30, 10/31  
Tutorial 8  Probabilistic models, Bayesian Inference, Pyro  11/6, 11/7  [ipynb] 
Tutorial 9  Reinforcement Learning  11/13, 11/14  [Slides] 
Tutorial 10  Reinforcement Learning 2  11/20, 11/21  [Slides] [ipynb] 
Tutorial 11  Final exam Review  11/27, 11/28  [Slides] 
The easiest option is probably to install everything yourself on your own machine.
If you don't already have python 3, install it.
We recommend some version of Anaconda (Miniconda, a nice lightweight conda, is probably your best bet). You can also install python directly if you know how.
Optionally, create a virtual environment for this class and step into it. If you have a conda distribution run the following commands:
conda create name csc2515 source activate csc2515
Use pip
to install the required packages
pip install scipy numpy autograd matplotlib jupyter sklearn
All the required packages are already installed on the Teaching Labs machines.