CSC 311 Fall 2021: Introduction to Machine Learning

Overview

Machine learning (ML) is a set of techniques that allow computers to learn from data and experience, rather than requiring humans to specify the desired behaviour by hand. ML has become increasingly central both in AI as an academic field, and in industry. This course provides a broad introduction to some of the most commonly used ML algorithms. It also serves to introduce key algorithmic principles which will serve as a foundation for more advanced courses, such as CSC412/2506 (Probabilistic Learning and Reasoning) and CSC413/2516 (Neural Networks and Deep Learning).

We start with nearest neighbors, the canonical nonparametric model. We then turn to parametric models: linear regression, logistic regression, softmax regression, and neural networks. We then move on to unsupervised learning, focusing in particular on probabilistic models, but also principal components analysis and K-means. Finally, we cover the basics of reinforcement learning.

Announcements

• The course information handout (syllabus) is available.

Where and When

Unfortunately, due to the evolving COVID-19 situation, the specific class format is subject to change. As of this writing (9/2), we are required to have an in-person component to this class; we've decided to have in-person lectures. Fortunately, we're not required to force you to show up (with the exception that the final exam may need to be in person - more on that below). We are designing the course so that you do not need to show up in person if you don't want to, and so that not showing up will not put you at a disadvantage. We'll do everything we can to maintain a safe environment while following the letter of University policy.

The first two weeks of class will be entirely virtual, to accommodate international arrivals. After that point, the current plan is for lectures to be held in-person. As this would require in-person gatherings well beyond the size currently allowed by Ontario public health authorities for most purposes, we can make few assumptions about what will be permitted and safe by the time in-person instruction is set to begin. Please continue monitoring Quercus and this course web page for further updates.

Tutorials and office hours will be held virtually throughout the term. Students are encouraged to attend both lecture and tutorial each week, but attendance won't be taken.

We will also accommodate students who are unable to attend in person. Specifically, one of the four lecture sections will be held virtually. Additionally, students will have access to the lecture videos from Fall 2020, which follows roughly the same schedule as this year's class.

Most terms, we allow students to attend sections other than their assigned ones. However, this year, if you choose to attend in-person, you must attend your assigned section, since overcrowding during a pandemic would create an unsafe situation. Note that different sections are held simultaneously in different rooms, so make sure you are in the correct room. It's likely that many of you will want to switch between virtual and in-person formats. Here are the policies on that:

• If you are assigned to the virtual section and would like to switch to an in-person section, then please let us know by Sept. 16 (through a form which will be communicated through Quercus).
• If you are assigned to an in-person section and would like to attend the virtual section, then please let us know by Sept. 16 through the same form. The reasons are (1) so that we can manage capacity in the in-person sections to accommodate the former group, and (2) so we can make sure to have a Zoom license with sufficient capacity.
• If you are assigned to one in-person section and would like to switch to a different in-person section, let us know by Sept. 16 through the same form. This will require a justification, such as a scheduling conflict with another course.
• Auditing is not allowed this term.

Specifics about online delivery will be sent to enrolled students through Quercus.

Modulo capacity constraints for the in-person sections, it will be up to you to decide whether the in-person lecture experience is worth the risk. We can't make the decision for you, but do consider that (as of this writing) most in-person gatherings of more than 25 people are banned, and that not only is the lecture a crowded indoor setting, but most of your fellow students will have been attending lots of other in-person lectures. In keeping with the theme of the course, where we use data to make decisions under uncertainty, you may find the MicroCovid Calculator helpful in reasoning about your own risk tolerance.

Course videos and materials belong to your instructor, the University, and/or other source depending on the specific facts of each situation, and are protected by copyright. In this course, you are permitted to download session videos and materials for your own academic use, but you should not copy, share, or use them for any other purpose without the explicit permission of the instructor.

For questions about recording and use of videos in which you appear please contact the instructors.

Teaching Staff

Communication

We will use Piazza for the course forum.

• If your question is about the course material and doesn't give away any hints for the homework, please post to Piazza so that the entire class can benefit from the answer.
• If you have questions that may give away homework answers, please post privately to Piazza.
• For course administration matters (homework extensions, missed exams, etc.) please email the instructors (see below).

All office hours will take place via Gather Town. Details will be communicated via Quercus.

Instructors

Teaching Assistants

Marking Scheme

We will use the following marking scheme:

• 3 homework assignments (35%, weighted equally)
• minor assignments for embedded ethics unit (5%)
• Good faith effort = full credit
• project (20%)
  • Due 12/3.
• 2 online tests (40%)
  • 1-hour online midterm test. Held during tutorial time on 10/21 or 10/22.
  • 2-hour online final exam, 14:00-16:00 on Friday, 12/10.
  • Weighting: higher of (15% midterm, 25% final) or (10% midterm, 30% final).

Homeworks

Homeworks will generally be due at 11:59pm on Wednesdays, and submitted through MarkUs. Please see the course information handout for detailed policies (marking, lateness, etc.). The detailed schedule will be posted soon.

Final Project

Schedule

This is a tentative schedule, which will likely change as the course goes on.

Suggested readings are optional; they are resources we recommend to help you understand the course material. All of the textbooks listed below are freely available online.

Bishop = Pattern Recognition and Machine Learning, by Chris Bishop.
ESL = The Elements of Statistical Learning, by Hastie, Tibshirani, and Friedman.
MacKay = Information Theory, Inference, and Learning Algorithms, by David MacKay.
Barber = Bayesian Reasoning and Machine Learning, by David Barber.
Sutton and Barto = Reinforcement Learning: An Introduction, by Sutton and Barto.

Computing Resources

1. The easiest option is probably to install everything yourself on your own machine.

   1. If you don't already have python 3, install it.

      We recommend some version of Anaconda (Miniconda, a nice lightweight conda, is probably your best bet). You can also install python directly if you know how.

   2. Optionally, create a virtual environment for this class and step into it. If you have a conda distribution run the following commands:

          conda create --name csc311
          source activate csc311

   3. Use pip to install the required packages

          pip install scipy numpy autograd matplotlib jupyter sklearn

2. All the required packages are already installed on the Teaching Labs machines.