CSC 411 Fall 2018

Machine Learning and Data Mining

Overview

Machine learning is a set of techniques that allow machines to learn from data and experience, rather than requiring humans to specify the desired behavior by hand. Over the past two decades, machine learning techniques have become increasingly central both in AI as an academic field, and in the technology industry. This course provides a broad introduction to some of the most commonly used ML algorithms. It also serves to introduce key algorithmic principles which will serve as a foundation for more advanced courses, such as CSC412/2506 (Probabilistic Learning and Reasoning) and CSC421/2516 (Neural Networks and Deep Learning).

The first half of the course focuses on supervised learning. We begin with nearest neighbours, decision trees, and ensembles. Then we introduce parametric models, including linear regression, logistic and softmax regression, and neural networks. We then move on to unsupervised learning, focusing in particular on probabilistic models, but also principal components analysis and K-means. Finally, we cover the basics of reinforcement learning.

Policies (marking, prerequisites, etc.)

See the course information handout.

Where and When

There are four sections of the course. Since all sections are fully subscribed, please attend the one you are registered for.

Starting Monday, 11/19, all tutorials will be held in the main lecture room for the corresponding section.

Teaching Staff

• Instructors: Roger Grosse, Amir-massoud Farahmand, and Juan Carrasquilla
• Office hours:
  • Juan: Mondays 12-1pm in BA 2283
  • Amir-massoud: Mondays 1-2pm in BA 2283
  • Roger: Mondays 5-7pm in PT 290F
  • TA office hours will be announced for each homework assignment.
• Head TA: James Lucas
• TAs: Aryan Arbabi, Jesse Bettencourt, Elliot Creager, Chris Cremer, Mohammad Firouzi, Jonathan Lorraine, David Madras, Bret Nestor, Punit Shah, and Guodong Zhang
• Staff e-mails:
  • TAs and instructor: csc411.f18.staff [at] gmail.com
  • Instructors and head TA only: csc411.f18.instructors [at] cs.toronto.edu
  • Please do not contact us at our personal e-mails.
• We will use Piazza for the course forum. If your question is about the course material and doesn't give away any hints for the homework, please post to Piazza so that the entire class can benefit from the answer.

Homeworks

Most weekly homeworks will be due on Wednesdays at 11:59pm. Please see the course information handout for detailed policies (marking, lateness, etc.).

Tests

The course will have a midterm and a final exam. Everybody is required to take both, including graduate students.

The midterm will be held from 6-7pm on Friday, October 19.

• If you are taking the test at the main time, then you are assigned to one of the following rooms based on the first letter of your last name. Pay attention to which room you're assigned, since we're up against the capacity limit in all of these rooms.
  • A-C: GB404
  • D-J: GB405
  • K-N: BA1130
  • O-R: HA401
  • S-X: HA403
  • Y-Z: HA410
• The alternative time is 8-9pm on Thursday, October 18, in EX310. Because we scheduled this time based on all of the conflicts you reported, we assume all of you can make either the standard time or the alternate time.
• Because the alternate room has limited capacity, you must notify us (by e-mailing the staff mailing list) if you need to use the alternative time.
• The test will cover Lectures 1 through 9. You are not responsible for concepts introduced only in readings, tutorials, etc. However, we are likely to ask harder questions on things you've had the opportunity to practice in the homeworks.
• You may find it helpful to practice with past years' midterms (but note that they may cover a slightly different set of topics):
  • Fall 2017 midterm
  • Winter 2018 midterm and solutions
• TA office hours:
  • Tues, 10/16, 6-7pm, in BA3201
  • Wed, 10/17, 2-3pm, in BA2283
  • Thurs, 10/18, 2-3pm, in BA3201
  • Fri, 10/19, 4-5pm, in BA3289
• Questions: main version, alternate version, and Solutions

The final exam time will be from 7-10pm on Tuesday, December 11.

• Room locations:
  • csc411, last name A-SE: BN 2N
  • csc411, last name SH-Z: BN 2S
  • csc2515, all students: BN 3
  • BN = Clara Benson Building, 320 Huron St.
• Practice exams:
  • Fall 2017
  • Exams from past years can be found from the university libary. Click on Past Computer Science Exams Online, and then search for csc411.
  • More details in this slide

Lectures

Here is a tentative schedule, which will likely change as the course goes on. Each "Lecture" corresponds to 50 minutes, so each 2-hour lecture session will cover 2 of them.

Suggested readings are just that: resources we recommend to help you understand the course material. They are not required, i.e. you are only responsible for the material covered in lecture.

ESL = The Elements of Statistical Learning, by Hastie, Tibshirani, and Friedman.
MacKay = Information Theory, Inference, and Learning Algorithms, by David MacKay.
Barber = Bayesian Reasoning and Machine Learning, by David Barber.
Bishop = Pattern Recognition and Machine Learning, by Chris Bishop.
Sutton and Barto = Reinforcement Learning: An Introduction, by Sutton and Barto.

Tutorials

Final Project

Paper Readings

1. P. Viola and M. Jones. Rapid object detection using a boosted cascade of simple features. CVPR 2001. [pdf]
2. A. Krizhevsky, I. Sutskever, and G. E. Hinton. ImageNet classification with deep convolutional neural networks. NIPS 2012. [pdf]
3. R. Salakhutdinov and A. Mnih. Probabilistic matrix factorization. NIPS 2007. [pdf]
4. B. A. Olshausen and D. J. Field. Sparse coding with an overcomplete basis set: a strategy employed by V1? Vision Research, 1997. [pdf]
5. V. Mnih et al. Human-level control through deep reinforcement learning. Nature, 2015. [article]
6. M. Hardt, E. Price, and N. Srebro. Equality of opportunity in supervised learning. NIPS 2016. [short version] [long version (optional)]

Computing Resources

1. The easiest option is probably to install everything yourself on your own machine.

   1. If you don't already have python 3, install it.

      We recommend some version of Anaconda (Miniconda, a nice lightweight conda, is probably your best bet). You can also install python directly if you know how.

   2. Optionally, create a virtual environment for this class and step into it. If you have a conda distribution run the following commands:

          conda create --name csc411
          source activate csc411

   3. Use pip to install the required packages

          pip install scipy numpy autograd matplotlib jupyter sklearn

2. All the required packages are already installed on the Teaching Labs machines.