STA314 Fall 2025: Statistical Methods for Machine Learning I

Overview

Machine learning (ML) is a set of techniques that allow computers to learn from data and experience, rather than requiring humans to specify the desired behaviour by hand. ML has become increasingly central both in statistics as an academic discipline, and in the data science industry. This course provides a broad introduction to commonly used ML methods, as well as the key statistical concepts underlying ML. It serves as a foundation for more advanced courses, such as STA414 (Statistical Methods for Machine Learning II).

We will cover statistical methods for supervised and unsupervised learning from data: training error, test error and cross-validation; classification, regression, and logistic regression; principal components analysis; stochastic gradient descent; decision trees and random forests; k-means clustering and nearest neighbour methods. Computational tutorials will support the efficient application of these methods.

Announcements

• Homework 1 is out.
• The syllabus is available.
• Announcements will be through Quercus and here, on the course website.

Course Information

The syllabus contains all of the course policies. Unless otherwise specified, the lectures, tutorials, office hours, midterm and final exam will be delivered in-person. There will be one mandatory midterm test held during the scheduled class time, and we will have a FAS proctored final examination.

Staff

Lectures, Tutorials, Office Hours

There are two sections of this course being offered this term. Office hours will be held during the third hour of the lecture time slot. You can attend any of the office hours, but please attend your assigned lecture and tutorial section. Room information should be available on ACORN.

Students enrolled in LEC0101 must enroll in one of TUT0101-0104. Students enrolled in LEC5101 must enroll in one of TUT0201-0204. Note that the Monday tutorial sections correspond to lectures given in the preceding week, as this term starts on a Tuesday.

Homework

The course will have four homeworks, due at 11:59pm on Mondays on the week that they are due. They will be submitted through Crowdmark on Quercus. We will be hosting TA office hours to help you prepare your assignments. This is a tentative schedule, and any changes will be announced.

Office hours are held at either the Sid. Smith Stats. Aid Centre or Zoom (see Quercus home page for the Zoom link)

Midterm and Final Exam

The course will have one midterm test held during the normal class time and a final exam proctored by FAS. The final exam schedules will be available on the A&S page. We will be hosting TA office hours to help you review for the tests and exams.

For both the midterm and final exam, you will be allowed to bring one double-sided aid-sheet (8.5" by 11").

You must take the tests with your assigned section, unless you have prior permission from the instructor. Please note, the lecture schedule on both days will be somewhat unusual; see details below.

Office hours are held at either the Sid. Smith Stats. Aid Centre or Zoom (see Quercus home page for the Zoom link)

Practice Midterms

Please be aware that these are from a different series of courses that may vary in difficulty and may have covered different material. Nevertheless, you should be able to get a sense for the style of questions that I may ask, and it will be very good practice to work through these.

• CSC411 F17 midterm.
• CSC411 W18 midterm with solutions.
• CSC411 F18 midterm 1 and CSC411 F18 midterm 2 with solutions.
• CSC311 F19 midterm with solutions. Just do Q1 and Q5.
• CSC311 F22 midterm A with solutions.
• CSC311 F22 midterm B with solutions.

Schedule

This is a preliminary schedule; it may change throughout the term. The suggested readings (see legend below) are completely optional, but recommended.

*Reading week is Oct 27–Oct 31, therefore the Nov 3 tutorial corresponds to lectures given on Oct 22.

Suggested References

• PRML = Pattern Recognition and Machine Learning, by Chris Bishop.
• ESL = The Elements of Statistical Learning, by Hastie, Tibshirani, and Friedman.
• LTFP = Learning Theory from First Principles, by Fancis Bach.
• Information Theory, Inference, and Learning Algorithms, by David MacKay.
• Bayesian Reasoning and Machine Learning, by David Barber.
• Reinforcement Learning: An Introduction, by Sutton and Barto.

Computing

Python Programming Language

For the homework assignments, we will use Python, and libraries such as NumPy, SciPy, and scikit-learn. We will not be expecting you to know advanced Python programming, however we will expect that you are able to do the following.

• navigate the file structure of a basic Python project,
• execute and read through Python scripts while understanding the distinction between code that executes and comments,
• basic Python programming, including calling and defining functions; assigning variables, evaluating expressions, the basics of loops, etc.
• read NumPy, SciPy, or scikit-learn documentation to understand their API.

Python Tutorials

There are a number of great Python tutorials on the web.

• Real Python has a variety of great tutorials to get you started. In particular, this NumPy tutorial.
• The official Python tutorial is quite detailed and comprehensive (has more than you'll need to know).

Using Python

There are a few options for running Python yourself.

• The easiest option is probably to install everything yourself on your own machine.

  • If you don’t already have python, install it. We recommend using Anaconda. You can also install python directly using the instructions here.

  • Optionally, create a virtual environment for this class and step into it. If you have a conda distribution run the following commands:

    conda create --name sta314
    source activate sta314

  • Use pip to install the required packages pip install scipy numpy autograd matplotlib jupyter sklearn
• You may also consider using UofT Jupyter Hub or Google Colab (if you have access to a Google account).