Machine learning (ML) is a set of techniques that allow computers to learn from data and experience, rather than requiring humans to specify the desired behaviour by hand. ML has become increasingly central both in AI as an academic field, and in industry. This course provides a broad introduction to some of the most commonly used ML algorithms. It also serves to introduce key algorithmic principles which will serve as a foundation for more advanced courses, such as CSC412/2506 (Probabilistic Learning and Reasoning) and CSC413/2516 (Neural Networks and Deep Learning).
We start with nearest neighbors, the canonical nonparametric model. We then turn to parametric models: linear regression, logistic regression, softmax regression, and neural networks. We then move on to unsupervised learning, focusing in particular on probabilistic models, but also principal components analysis and Kmeans. Finally, we cover the basics of reinforcement learning.
Each section of this course corresponds to one lecture and one tutorial time. Class will be held synchronously online every week, including a combination of lecture and tutorial exercises. Students are encouraged to attend both the lecture and tutorial each week. There will be two mandatory tests held during the scheduled class time.
Sections  Lecture Time  Tutorial Time 
LEC0101, LEC0102, LEC2001  Monday 111  Monday 34 
LEC0201, LEC0202, LEC2001  Thursday 46  Thursday 78 
Online delivery. Lectures will be delivered synchronously via Zoom, and recorded for asynchronous viewing by enrolled students. Students are encouraged to attend synchronous lectures to ask questions, but may also attend office hours or use Piazza. All information about attending virtual lectures, tutorials, and office hours will be sent to enrolled students through Quercus.
Course videos and materials belong to your instructor, the University, and/or other source depending on the specific facts of each situation, and are protected by copyright. In this course, you are permitted to download session videos and materials for your own academic use, but you should not copy, share, or use them for any other purpose without the explicit permission of the instructor. For questions about recording and use of videos in which you appear please contact your instructor.Juhan Bae  Roger Grosse  Chris Maddison  Silviu Pitis  
Office Hours  Thursday 24  Monday 13  Monday 68  Friday 1012 
Sections  LEC0102  LEC0201  LEC0101, LEC2001  LEC0202 
Email Instructors Only  csc311202009@cs.toronto.edu 
We will use Piazza for the course forum. If your question is about the course material and doesn't give away any hints for the homework, please post to Piazza so that the entire class can benefit from the answer.
Linear Algebra Review  Midterm 1 Prep  Midterm 2 Prep  
Office hours  Thu 10/8 89pm Fri 10/9 910am Tue 10/13 10:30am12:30pm Fri 10/16 1:303:30pm 
Fri 10/16 45pm Mon 10/19 1011am Tue 10/20 89pm Wed 10/21 23pm Thu 10/22 910am 
TBA  
Email TAs & Instructors  csc311202009tas@cs.toronto.edu 
Most weekly homeworks will be due at 11:59pm on Wednesdays, and submitted through MarkUs. Please see the course information handout for detailed policies (marking, lateness, etc.).
#  Out  Due  Materials  TA Office Hours 
1  9/17  9/30  [Handout] [Data] 
Fri 9/25 59pm Mon 9/28 911am Tues 9/29 10:30am12:30pm Wed 9/30 10:30am12:30pm 
2  10/1  10/14  [Handout] [Code & Data] [Code V2 & Data] 
Wed 10/07 46pm Fri 10/09 46pm Mon 10/12 911am Tues 10/13 79pm Wed 10/14 24pm 
3  10/15  11/4  [Handout] [Code] 
Wed 10/28 46pm Fri 10/30 46pm Mon 11/02 911am Tues 11/03 79pm Wed 11/04 24pm 
4  11/5  11/25  [Handout] [Handout v2] [Code & Data] 
Wed 11/18 46pm Fri 11/20 46pm Mon 11/23 911am Tues 11/24 79pm Wed 11/25 24pm 
The course will have two tests, each with a duration of 1 hour and held during the normal class time. The higher of the two marks will count for 15%, and the lower mark will count for 10%.
The lecture schedule on both days will be somewhat unusual; see details below. The reason for this is that some students will be in other time zones, and we wanted to make sure any time zone has at least one exam time which is at least acceptable.
You must take the test with your assigned section, unless you have prior permission from the instructor.
#  Thursday Section  Monday Section  Covers 
1  10/22 Test: 78pm Lecture: 46pm 
10/26 Test: 11amnoon Lecture: noon1pm, 34pm 
Up through Lecture 6 
2  12/3 Test: 78pm Lecture: 45pm 
12/7 Test: 11amnoon Lecture: noon1pm 
Up through Lecture 11 
This is a tentative schedule, which will likely change as the course goes on.
Suggested readings are optional; they are resources we recommend to help you understand the course material. All of the textbooks listed below are freely available online.
Bishop = Pattern Recognition and Machine Learning, by Chris Bishop.
ESL = The Elements of Statistical Learning, by Hastie, Tibshirani, and Friedman.
MacKay = Information Theory, Inference, and Learning Algorithms, by David MacKay.
Barber = Bayesian Reasoning and Machine Learning, by David Barber.
Sutton and Barto = Reinforcement Learning: An Introduction, by Sutton and Barto.
#  Dates  Topic  Materials  Suggested Readings 
1  9/10, 9/14 
Lecture: Introduction, Nearest Neighbours
Tutorial: Probability 
Lecture: [Slides] Tutorial: [Slides] 
ESL: 1, 2.12.3, 2.5 
2  9/17, 9/21 
Lecture: Linear Methods for Regression, Optimization
Tutorial: Linear Algebra Review 
Lecture: [Slides] Tutorial: [Slides] 
Bishop: 3.1 ESL: 3.1  3.2 Course notes: Linear Regression, Calculus 
3  9/24, 9/28 
Lecture: Logistic Regression, Multiclass Classification, Optimization
Tutorial: Optimization 
Lecture: [Slides] Tutorial: 
Bishop: 4.1, 4.3 ESL: 4.14.2, 4.4, 11 Course notes: Linear Classifiers, Training a Classifier 
4  10/1, 10/5 
Lecture: Neural Networks
Tutorial: PyTorch 
Lecture: [Slides] Tutorial: 
Bishop: 5.15.3 Course notes: Multilayer Perceptrons, Backpropagation 
5  10/08, 10/12 
Lecture: Decision Trees, BiasVariance Decomposition
Tutorial: TBA Note: Lecture will be held as usual on Thanksgiving. It will be recorded as usual, so you are welcome to watch the recording instead. 
Lecture: [Slides] Tutorial:

Bishop: 3.2 ESL: 2.9, 9.2 Course notes: Generalization 
6  10/15, 10/19 
Lecture: Bagging, Boosting
Tutorial: Midterm Review 
Lecture: [Slides] Tutorial: [Slides] 
ESL: 8.7, 10.110.5 
7  10/22, 10/26 
Lecture: Probabilistic Models
Tutorial: None (inclass test) 
Lecture: [Slides] 
ESL: 2.6.3, 6.6.3, 4.3.0 MacKay: 21, 23, 24 Course notes: Probabilistic Models 
8  10/29, 11/2 
Lecture: Probabilistic Models cont'd; Principal Component Analysis
Tutorial: Eigenvectors, PCA 
Lecture: [Slides] Tutorial: [Notes] 
Bishop: 12.1 
9  11/5, 11/16 
Lecture: PCA cont'd; Matrix Completion; Autoencoders
Tutorial: Final Project 
Lecture: [Slides] Tutorial: [Slides] [Colab] 
ESL: 14.5.1 
10  11/19, 11/23 
Lecture: kMeans, EM Algorithm
Tutorial: TBA 
Lecture: [Slides] Tutorial: [Slides] 
MacKay: 20 Bishop: 9 Barber: 20.120.3 Course notes: Mixture Modeling 
11  11/27, 11/30 
Lecture: Reinforcement learning
Tutorial: TBA 
[Lecture], [Tutorial]  Sutton and Barto: 3, 4.1, 4.4, 6.16.5 
12  12/3, 12/7 
Lecture: TBA
Tutorial: None (inclass test) 
[Lecture] 
The easiest option is probably to install everything yourself on your own machine.
If you don't already have python 3, install it.
We recommend some version of Anaconda (Miniconda, a nice lightweight conda, is probably your best bet). You can also install python directly if you know how.
Optionally, create a virtual environment for this class and step into it. If you have a conda distribution run the following commands:
conda create name csc311 source activate csc311
Use pip
to install the required packages
pip install scipy numpy autograd matplotlib jupyter sklearn
All the required packages are already installed on the Teaching Labs machines.