CSC321 Neural Networks and Machine Learning (UTM)

Winter 2020

Home
Piazza
MarkUs
Homework
Projects
Team
Midterm
Exam FAQ

Course Description

Machine learning is a powerful set of techniques that allow computers to learn from data rather than having a human expert program a behaviour by hand. Neural networks are a class of machine learning algorithm originally inspired by the brain, but which have recently seen a lot of success at practical applications. They're at the heart of production systems at companies like Google and Facebook for face recognition, speech-to-text, and language understanding.

This course gives an overview of both the foundational ideas and the recent advances in neural net algorithms. Roughly the first 2/3 of the course focuses on supervised learning -- training the network to produce a specified behaviour when one has lots of labelled examples of that behaviour. The last 1/3 focuses on unsupervised learning.

See the course information sheet for more information.

Course Staff and Contact

Pouria FewzeeLEC0101 (W 9am-11am)Office Hours: Wednesdays 12pm-2pm (MN5107)pouria [dot] fewzee [at] utoronto [dot] ca
Lisa Zhang (*Coordinator)LEC0102 (W 11am-1pm)Office Hours: Monday 12pm-2pm (DH3078)lczhang [at] cs [dot] toronto [dot] edu

Contact the course coordinator (Lisa) for all logistics related inquiries (e.g. medical notes, remark requests). There are other people in the University of Toronto community with the same name as Lisa. Please make sure that you are emailing the correct one.

Please use the Piazza message board for questions reqlated to course content.

All announcements will be made on Piazza and Quercus

Textbook

We will be using written notes by Prof. Roger Grosse, to be posted on the course website.

Tentative Schedule

The course schedule is tentative and subject to change.

Lecture 1 (Jan 8)
  • Introduction
  • Linear Regression
  • Slides
Tutorial 1 (Jan 7/8 Optional)
  • Review of Pre-requisite Homework 0
Course Notes (by Prof. Roger Grosse)
Recommended Review
Homework 0 (ungraded)
Lecture 2 (Jan 15)
  • Gradient Descent
  • Linear Classification
  • Slides
Tutorial 2 (Jan 14/15)Course Notes (by Prof. Roger Grosse)Homework 1
(Jan 16, 9pm)
Lecture 3 (Jan 22)
  • Linear Classification
  • k-Nearest Neighbours
  • Generalization
  • Slides
Tutorial 3 (Jan 21/22)Course Notes (by Prof. Roger Grosse)
Additional Materials
Homework 2
(Jan 23, 9pm)
Lecture 4 (Jan 29)
  • Multi-layer perceptrons
  • Backpropagation
  • Slides
Tutorial 4 (Jan 28/29)Course Notes (by Prof. Roger Grosse)
Additional Materials
Just for fun
Project 1
(Jan 30, 9pm)
Lecture 5 (Feb 5)
  • Distributed Representations
  • Optimization
  • Generalization
  • Slides
Tutorial 5 (Feb 4/5)Course Notes (by Prof. Roger Grosse)Homework 3
(Feb 6, 9pm)
Lecture 6 (Feb 12)Tutorial 6 (Feb 11/12)
  • Optimization, Learning Curves, (and some CNN) [html] [pdf] [ipynb]
Course Notes (by Prof. Roger Grosse)Project 2
(Feb 13, 9pm)
Reading Week.Midterm Office Hours
  • Pouria (Instructor) Wednesday Feb 19th 12pm-2pm MN5107
  • Siva (TA) Friday Feb 21st 3pm-7pm MN3100
  • Lisa (Instructor) Monday Feb 24th 12pm-2pm DH3078
  • Wan (TA) Tuesday Feb 25th 5pm-7pm DH 2010
Lecture 7 (Feb 26)
  • Midterm (50 min)
  • Object Detection, AlexNet, VGG
  • Transfer Learning
  • Fully-Convolutional Networks
  • Slides
No tutorials this week
Wan will hold mditerm office hours on Tues Feb 25th 5pm-7pm in DH2010 instead
Course Notes (by Prof. Roger Grosse)Project 3 Data
(Feb 24, 9pm)
Lecture 8 (Mar 4)
  • Transpose Convolutions
  • Image Autoencoders
  • Slides
Tutorial 8 (Mar 3/4)Course Notes
Useful Resources
Lecture 9 (Mar 11)
  • Optimizing the Input
  • Adversarial Examples
  • Recurrent Neural Networks
  • Slides
Tutorial 9 (Mar 10/11)Course Notes
Course Notes (by Prof. Roger Grosse)
Homework 4
(Mar 12, 9pm)
Lecture 10 (Mar 18)
  • GloVe: Word Embeddings Revisisted
  • Reccurent Neural Networks for Classification
  • Gradient Explosion/Vanishing
  • Long Short-Term Memory
  • Slides
Tutorial 10 (Mar 17/18)
  • Cumulative Review (weeks 4-9)
Course Notes
Course Notes (by Prof. Roger Grosse)
Project 3
(Mar 19 23, 9pm)
Lecture 11 (Mar 25)
  • Skip Connections
  • Residual Networks
  • Attention
  • Slides
Tutorial 11 (Mar 24/25)Course Notes (by Prof. Roger Grosse)Homework 5
(Mar 26 30, 9pm)
Lecture 12 (Apr 1)
  • Generative Adversarial Networks
  • Slides
Tutorial 12
  • Exam Review
Course Notes
Course Notes (by Prof. Roger Grosse)
Project 4
(Apr 2 8, 9pm)
Final Examination Schedule