CSC411 is an undergraduate course which serves as a broad introduction to Machine Learning. We will cover a range of supervised and unsupervised learning algorithms. The topics covered will include linear models, non-parametric models, neural networks, ensemble methods, and reinforcement learning.
You will be expected to be familiar with basic probability theory and statistics (STA247/255/257). For example, you should be familiar with common probability distributions (Gaussian, Poisson, etc.). You should also be familiar with college-level linear algebra and calculus (MAT135/137/157). Experience with programming will be valuable for the assignments (CSC263/265); especially knowledge of Python.
| Section 1 | Section 2 | Section 3 | Section 4 | |
|---|---|---|---|---|
| Instructor | Ethan Fetaya | Emad Andrews | James Lucas | |
| Lecture Time | Monday 11am-1pm | Wednesday 11am-1pm | Thursday 4pm-6pm | Friday 11am-1pm |
| Lecture Location | AH 400 | OI 2212 | MP 103 | RW 117 |
| Office Hours | Tuesday 11am-1pm | Wednesday 9am-10am | Thursday 6pm-8pm | Friday 9am-10:45am |
| OH Location | Pratt 290A | Pratt 290A | BA3219 | BA3219 |
| Tutorial Times | Monday 3pm-4pm | Wednesday 3pm-4pm | Thursday 6pm-7pm | Friday 3pm-4pm |
| Tutorial Location | AH 400 | OI 2212 | MP 103 | RW 117 |
A minimum of 30% will need to be scored on the final exam to ensure a passing grade for this course.
Detailed breakdown of course homework and exams here.
We will be awarding a small percentage of total marks (5%) for reading classic research papers associated with the topics covered during lectures. This is intended to encourage independent research - an extremely valuable tool in a field which moves as quickly as machine learning. These points will be awarded through an honour system.
External reading will not be necessary to cover all course material. However, we will suggest reading to supplement lecture material. We will recommend sections from the following books:
Almost all questions should go through Piazza.
You can also email the course TAs.
Otherwise you can email the instructors.
Both undergraduate and graduate students must sit the midterm exam. The midterm exam will be worth 20% of your final course grade.
The midterm exam will take place from 7:10pm - 8:10pm on Friday Oct. 20th for all four sections. The midterm exam will be located in MS3153(surnames A-L)/MS3154(surnames M-Z).
The midterm exam will cover content up to and including Lecture 8.
All undergraduate students enrolled in CSC411 must sit the final exam. The final exam will be worth 30% of your final course grade. A minimum of 30% will need to be scored on the final exam to ensure a passing grade for this course.
The date of the final exam is December 20th.
The schedule given is tentative and may change slightly during the course.
| Lecture | Topic | Slides | Reading |
|---|---|---|---|
| Lecture 1 | Introduction | Handout | Bishop, Sections 1.0 - 1.1 |
| Lecture 2 | Linear Regression | Handout | Bishop, Section 3.1 |
| Tutorial 1 | Probability Review | Handout | |
| Lecture 3 | Linear Classification | Handout | Bishop, Sections 4 - 4.1.3 |
| Lecture 4 | Logistic Regression | Handout | Bishop, Sections 4.2.4 - 4.3.2Murphy, Sections 8.1 - 8.3.2 |
| Tutorial 2 | Gradient Descent | Handout - Demo | |
| Lecture 5 | Nearest Neighbour | Handout | Bishop, Section 2.5Murphy, Section 1.4.2 |
| Lecture 6 | Decision Trees | Handout - Extra Notes | Bishop, Section 14.4Murphy, Sections 16.2-16.2.3 |
| Tutorial 3 | k-NN and Decision Trees | Handout | |
| Lecture 7 | Multi-class Classification | Handout | Bishop, Sections 4-4.1.2 and 4.3.4 |
| Lecture 8 | Probabilistic Classifiers I | Handout | Bishop, Sections 4.2.2 - 4.2.3 |
| Tutorial 4 | Naive Bayes | Handout - Demo | |
| Lecture 9 | Probabilistic Classifiers II | Handout | Murphy, Sections 4.2-4.2.5 |
| Lecture 10 | Neural Networks I | Handout | Bishop, Sections 5-5.3 |
| Tutorial 5 | Neural Networks | Handout | |
| Lecture 11 | Neural Networks II | Handout | |
| Lecture 12 | PCA | Handout | Murphy, Sections 12.2-12.2.3 (*and beyond) |
| Tutorial 6 | Mid-term Review | Handout | |
| Lecture 13 | t-SNE | Handout | See reading assignment 3 |
| Lecture 14 | Clustering | Handout | Bishop, Sections 9-9.1 |
| Tutorial 7 | PCA | Handout - Demo | |
| Lecture 15 | Mixture of Gaussians (EM I) | Handout | Bishop, Sections 9.2-9.3.2 |
| Lecture 16 | EM Algorithm II | ||
| Tutorial 8 | Clustering | Handout | |
| Lecture 17 | SVM | Handout | Bishop, 7.1-7.1.3Murphy, parts of Chapter 14 |
| Lecture 18 | Kernels | Handout | |
| Tutorial 9 | SVM | Handout | |
| Lecture 19 | Ensembles I | Handout | Bishop Section 14.3Murphy Sections 16.2.4-16.2.5 |
| Lecture 20 | Ensembles II | ||
| Tutorial 10 | Gradient Boosting | Handout | |
| Lecture 21 | Reinforcement Learning I | Handout | In Slides |
| Lecture 20 | Reinforcement Learning II | ||
| Tutorial 11 | Reinforcement Learning | Handout | |
| Lecture 23 | Learning Theory I* | Handout | |
| Lecture 24 | Learning Theory II* |
*Not examined
Solutions to a selection of the assignment questions can be found here: Answers
| Release Date | Due Date | Document | Materials | |
|---|---|---|---|---|
| Assignment 1 | Sep. 21st | Oct. 5th | Document | Starter code: Q1 - Q2 - Q3 |
| Assignment 2 | Oct. 24th | Nov. 12th | Document | Data: Digits Starter Code: tarball |
| Assignment 3 | Nov. 19th | Dec. 6th | Document | Starter code: Q1 - Q2 |
You are not expected to fully understand these reading assignments and you will not be tested on any of the material within them. However, you should have the necessary knowledge from the course to understand the key ideas. We are proposing these reading assignments to help you develop important skills which will help you use machine learning independently later in your careers - whether you are a grad student or working in industry. On that note, reading papers is difficult and this is to be expected. To help you work through this process we strongly recommend you first read these notes from S. Keshav. These suggest a three-pass approach which can make digesting a paper-format report much easier.
| Release Date | Paper | |
|---|---|---|
| Reading 1 | Sep. 30th | On Discriminative vs. Generative Classifiers: A comparison of logistic regression and naive Bayes - Andrew Ng and Michael Jordan |
| Reading 2 | Oct. 16th | Regression Shrinkage and Selection via the Lasso - Robert Tibshirani |
| Reading 3 | Oct. 19th | Vizualizing Data using t-SNE - Laurens van der Maaten and Geoffrey Hinton |
| Reading 4 | Nov. 19th | Pegasos: Primal Estimated sub-GrAdient SOlver for SVM - Shalev-Schwartz et al. |
| Reading 5 | Dec 6th | A Theory of the Learnable - L.G. Valiant |
Grad students must submit a course project in lieu of a final exam. This project will be worth the equivalent grade percentage of 30%. The project guidelines explain in detail the expectations and grading rubric: project guidelines.
Important information: