CSCC11: Introduction to Machine Learning

Calendar: CSCC11 Introduction to Machine Learning

Below is a tentative calendar for the the course lectures and tutorials. This will be subject to change as we make our way through the course.

The assignment handouts, and the due dates for assignments are also given here. Please do not expect answers to questions about assignments within 20 hours of the assignment deadline.

NOTE that the required readings, including background readings and the online lecture notes, are available from the course lecture notes. In addition chapters of the online lecture notes, we also make reference to specific sections of well known textbooks that are listed on the textbook page .

Date Topic Readings & Demos Course Work

Week 1,
Jan 8-12,
2024 Lectures:

Introduction to Machine Learning

Linear Regression

Videos:

LinReg1, LinReg2, LinReg3, LinReg4

Corrections

No tutorial in week 1 Notes: Chapters 1, 2

Bishop (optional): Sections 1.5.5, 3.1-3.1.3, 5.2.3-5.2.4

Week 2,
Jan 15-19,
2024 Lectures:

Basis-Fn Regression (polynomials, RBFs)

Regularized Regression

KNN Regression

Videos:

basisFnReg1, basisFnReg2, basisFnReg3

ridgeReg

kNNreg

Tutorial:

Linear regression, Loss Functions, and Quadratics

Demos

Notes: Chapters 2, 3, and 4

numpy_basics.ipynb
linear_regression.ipynb
Assignment 1: handout, code & data starter

Week 3,
Jan 22-26,
2024 Lecture:

Probability

MAP, ML and Bayes Estimates

MAP Regression

Videos:

probEstimation1, probEstimation2, probEstimation3

Corrections
Tutorial:

Multi-dimensional Gaussian distributions

Diagonalization

Products, Marginals, and Conditionals

Notes: Chapters 5, 6, and 7

Bishop (optional): Sections 1.2-1.2.5

gaussians.ipynb

Week 4,
Jan 29-Feb 2,
2024 Lecture:

Introduction to Classification (Decision Boundaries)

kNN Classifiers

Decision Trees/Forests

Videos:

classification Intro1, classification Intro2

knn classification

decision tree1, decision tree2, decision tree3

Tutorial:

Probability and Bayes' rule

Estimating Gaussians (section 7.4)

Entropy, conditional entropy

Assignment 1 guidance

Notes: Chapters 7, 8, and 9

Bishop (optional): Sections 1.5-1.6, 4.1.1-4.1.3, 4.2, 14.4

Hastie et al (optional): Section 9.2, 13.3 Assignment 1 written work due Feb 1, 11:59pm.

Week 5,
Feb 5-9,
2024 Lecture:

Decision Trees/Forests

Class-Conditional Models

Naive Bayes

Videos:

decision trees3 / random forests

CC models1, CC models2

naive bayes1

Tutorial:

Bagging and Random Forests

Hyper-parameter selection for Random Forests

Notes: Chapters 8, 9

Bishop: Sections 4.2, 4.3.1-4.3.2 Assignment 1 programming exercises due Feb 8, 11:59pm

Practice Term Test

Week 6,
Feb 12-16,
2024 Lecture:

Naive Bayes

Logistic Regression

Generative vs Discriminative Models

Videos:

naive bayes2, naive bayes3

logistic reg1, logistic reg2, logistic reg3, logistic reg4

Tutorial:

Solutions to midterm test 1 and A1

Optimization, gradient descent

Notes: Chapters 9 - 10

Bishop: Sections 4.3.2, 4.3.4 Assignment 2: handout, code & data starter

Midterm on Monday Feb 12, 5-6pm in HLB101

Week 7,
Feb 26-Mar 1,
2024 Lecture:

Generative vs Discriminative Models

Cross Validation

Bayesian Methods

Bayesian Regression

Videos:

generative/discriminative models

Cross Validation

Bayesian methods

Bayes regression 1, Bayes regression 2
Tutorial:

Optimization (gradient descent, line search, stochastic gradient descent

Notes: Chapters 10 - 12

Bishop: Sections 3.3

Hastie et al: Sections 7.10

Week 8,
Mar 4-8,
2024 Lecture:

Bayesian Regression (model averaging)

Bayesian Model Selection

Videos:

Bayesian regression 3

Bayesian Model Selection 1, Model Selection 2

Tutorial:

A2 Written Solutions

Bayesian Methods/Regression

Notes: Chapters 11, 13 (13.3, 13.4 are optional)

Bishop: Sections 3.4

Hastie et al: Sections 7.1, 7.7 Assignment 3 theory questions due Monday

Assignment 3 coding questions due Thursday

Week 9,
Mar 11-15,
2024 Lecture:

Introduction to Unsupervised Learning

Principal Component Analysis

Videos:

PCA Intro

PCA2, PCA3, PCA4

Tutorial:

Solutions to Assignment 2

Bayesian Regression

Notes: Chapter 14

Bishop: Sections 12.1, 12.2.1

Hastie et al: 14.5.1 Midterm on Friday, Mar 15, 7-8pm, in HLB101.

Week 10,
Mar 18-22,
2024 Lecture:

Probabilistic PCA

Clustering: K-means, K-means++

Videos:

PPCA1, PPCA2

kMeans1, kMeans2, kMeans3, kMeans4

Tutorial:

Monte Carlo sampling

Sampling from categorical distributions

Principal Component Analysis (demo)

Notes: Chapters 14,16 (16.3, 16.4 are optional)

Bishop: Sections 11.0 - 11.1.2

Hastie et al: Sections 13.2, 14.3.6 - 14.3.12

Assignment 3: handout, code & data starter

Week 11,
Mar 25-29,
2024 Lecture:

Gaussian mixture models

Expectation-Maximization Alg.

Videos:

Gaussian Mixture Models

GMM2, GMM3

GMM4, GMM5, GMM6

Tutorial:

Solutions to term test 2

PPCA

Notes: Chapters 15, 16 (16.5.4 is optional)

Hastie et al: Sections 8.5

Week 12,
Apr 1-5,
2024 Lecture:

Exam Review

Discussion about AI/ML and research

Tutorial:

Help with A3

PPCA

Assignment 3 due Apr 8

Date	Topic	Readings & Demos	Course Work
Week 1, Jan 8-12, 2024	Lectures: Introduction to Machine Learning Linear Regression Videos: LinReg1, LinReg2, LinReg3, LinReg4 Corrections No tutorial in week 1	Notes: Chapters 1, 2 Bishop (optional): Sections 1.5.5, 3.1-3.1.3, 5.2.3-5.2.4
Week 2, Jan 15-19, 2024	Lectures: Basis-Fn Regression (polynomials, RBFs) Regularized Regression KNN Regression Videos: basisFnReg1, basisFnReg2, basisFnReg3 ridgeReg kNNreg Tutorial: Linear regression, Loss Functions, and Quadratics Demos	Notes: Chapters 2, 3, and 4 numpy_basics.ipynb linear_regression.ipynb	Assignment 1: handout, code & data starter
Week 3, Jan 22-26, 2024	Lecture: Probability MAP, ML and Bayes Estimates MAP Regression Videos: probEstimation1, probEstimation2, probEstimation3 Corrections Tutorial: Multi-dimensional Gaussian distributions Diagonalization Products, Marginals, and Conditionals	Notes: Chapters 5, 6, and 7 Bishop (optional): Sections 1.2-1.2.5 gaussians.ipynb
Week 4, Jan 29-Feb 2, 2024	Lecture: Introduction to Classification (Decision Boundaries) kNN Classifiers Decision Trees/Forests Videos: classification Intro1, classification Intro2 knn classification decision tree1, decision tree2, decision tree3 Tutorial: Probability and Bayes' rule Estimating Gaussians (section 7.4) Entropy, conditional entropy Assignment 1 guidance	Notes: Chapters 7, 8, and 9 Bishop (optional): Sections 1.5-1.6, 4.1.1-4.1.3, 4.2, 14.4 Hastie et al (optional): Section 9.2, 13.3	Assignment 1 written work due Feb 1, 11:59pm.
Week 5, Feb 5-9, 2024	Lecture: Decision Trees/Forests Class-Conditional Models Naive Bayes Videos: decision trees3 / random forests CC models1, CC models2 naive bayes1 Tutorial: Bagging and Random Forests Hyper-parameter selection for Random Forests	Notes: Chapters 8, 9 Bishop: Sections 4.2, 4.3.1-4.3.2	Assignment 1 programming exercises due Feb 8, 11:59pm Practice Term Test
Week 6, Feb 12-16, 2024	Lecture: Naive Bayes Logistic Regression Generative vs Discriminative Models Videos: naive bayes2, naive bayes3 logistic reg1, logistic reg2, logistic reg3, logistic reg4 Tutorial: Solutions to midterm test 1 and A1 Optimization, gradient descent	Notes: Chapters 9 - 10 Bishop: Sections 4.3.2, 4.3.4	Assignment 2: handout, code & data starter Midterm on Monday Feb 12, 5-6pm in HLB101
Week 7, Feb 26-Mar 1, 2024	Lecture: Generative vs Discriminative Models Cross Validation Bayesian Methods Bayesian Regression Videos: generative/discriminative models Cross Validation Bayesian methods Bayes regression 1, Bayes regression 2 Tutorial: Optimization (gradient descent, line search, stochastic gradient descent	Notes: Chapters 10 - 12 Bishop: Sections 3.3 Hastie et al: Sections 7.10
Week 8, Mar 4-8, 2024	Lecture: Bayesian Regression (model averaging) Bayesian Model Selection Videos: Bayesian regression 3 Bayesian Model Selection 1, Model Selection 2 Tutorial: A2 Written Solutions Bayesian Methods/Regression	Notes: Chapters 11, 13 (13.3, 13.4 are optional) Bishop: Sections 3.4 Hastie et al: Sections 7.1, 7.7	Assignment 3 theory questions due Monday Assignment 3 coding questions due Thursday
Week 9, Mar 11-15, 2024	Lecture: Introduction to Unsupervised Learning Principal Component Analysis Videos: PCA Intro PCA2, PCA3, PCA4 Tutorial: Solutions to Assignment 2 Bayesian Regression	Notes: Chapter 14 Bishop: Sections 12.1, 12.2.1 Hastie et al: 14.5.1	Midterm on Friday, Mar 15, 7-8pm, in HLB101.
Week 10, Mar 18-22, 2024	Lecture: Probabilistic PCA Clustering: K-means, K-means++ Videos: PPCA1, PPCA2 kMeans1, kMeans2, kMeans3, kMeans4 Tutorial: Monte Carlo sampling Sampling from categorical distributions Principal Component Analysis (demo)	Notes: Chapters 14,16 (16.3, 16.4 are optional) Bishop: Sections 11.0 - 11.1.2 Hastie et al: Sections 13.2, 14.3.6 - 14.3.12	Assignment 3: handout, code & data starter
Week 11, Mar 25-29, 2024	Lecture: Gaussian mixture models Expectation-Maximization Alg. Videos: Gaussian Mixture Models GMM2, GMM3 GMM4, GMM5, GMM6 Tutorial: Solutions to term test 2 PPCA	Notes: Chapters 15, 16 (16.5.4 is optional) Hastie et al: Sections 8.5
Week 12, Apr 1-5, 2024	Lecture: Exam Review Discussion about AI/ML and research Tutorial: Help with A3 PPCA		Assignment 3 due Apr 8