---

ANNOUNCEMENTS:

I have added a separate submission portal on UTORSubmit for Question 5 of assignment 3. (If you have already submitted question 5 to the portal for assignment 3, you can just leave it there, but any updates should be submitted to the new portal. Any material in the new portal will be assumed to be your definitive submission for question 3.)

The deadline for Question 5 of Assignment 3 is extended until April 24 at 11pm. However, if you need your final course grade by April 21 (for June graduation), Question 5 must be handed in by April 18 at 11pm. Questions 1-4 are still due on April 16 at 11pm for everyone.

Marked Assignment 2s can be picked up from the TA (BA6135) on April 20 (not 19) from noon to 2pm. Please email him before you come and give your approximate arrival time (shuangjian [dot] zhang [at] mail [dot] utoronto [dot] ca).

The deadline for Question 5 of Assignment 3 will be extended. Details to come.

Question 4(f) on Assignment 3 has been clarified.

In Question 4 of Assignment 3, you may assume that K is symmetric and positive definite. You may also assume that equation (1) has a minimum.

Questions 4(e) and (f) on Assignment 3 have been changed. (They are now more specific and should be easier to answer.)

In Question 5 of Assignment 3, you should be able to obtain a cross-validation error of less than 0.02

The deadline for Assignment 3 has been extended to Sunday April 16 at 11pm.

Assignment 3 is now complete. No more questions will be added.

The last class will be on April 6.

The first four questions of Assignment 3 are niw available, below.

Assignment 2 is now complete. No more questions will be added.

A programming problem has been added to Assignment 2. The due date and submission instructions are described at the start of the assignment sheet.

A programming question will be added to Assignment 2, but you will have an extra week to hand it in. All other questions are due on March 23.

The last class will be on April 6.

The role of symmetry in Question 5 of Assignment 2 has been clarified. Please see the assignment sheet.

The first five questions of Assignment 2 have been posted, below.

The first three questions of Assignment 2 have been posted, below.

Assignment 2 is due on Thursday March 23.

Computer programs for all assignments should be submitted electronically at the UTORSubmit website, as described below.

All questions have now been added to Assignment 1.

The number of test points in Question 4(a) of Assignment 1 as been reduced from 20,000 to 2000.

Questions 2 and 3 in Assignment 1 have been slightly elaborated.

An introduction to Question 4 has been added to Assignment 1.

A third question has been added to Assignment 1.

As discussed in class, Question 2 of Assignment 1 should refer to Question 7.13 in the text, not Question 7.10. (See the corrected assignment, below)

The first two questions of Assignment 1 are now available, below. More questions will be added shortly.

---

COURSE DESCRIPTION:

The introduction of Support Vector Machines (SVMs) in the 1990s led to an explosion of applications and deepening theoretical analysis that have established SVMs as one of the standard tools for machine learning and data mining. This course provides a comprehensive introduction to SVMs and other kernel methods, including theory, algorithms and applications. Topics covered will be selected from the following: support vector classification and regression; novelty detection and feature extraction; non-linear dimensionality reduction; reproducing kernel maps; regularization; statistical learning theory and robust estimation; convex optimization and implementation; kernel design and applications. Homework assignments will be a mix of theory and programming.

---

BASIC INFORMATION:

Research Area 12 (Machine Learning), Methodology 2 (Continuous Models).

Lectures: Thursday 2-4pm. The first class will be on January 12. The last class will be on April 6.

Location: OI 4-414 (OISE, 252 Bloor St W)

Expected work: Three assignments, which will be a mix of theory and programming

Prerequisites: Linear algebra, vector calculus, basic probability, and a willingness to program in Python. A previous course in machine learning is not required but would be helpful. Mathematical maturity will be assumed.

INSTRUCTOR:

Anthony Bonner

email: my last name [at] cs [dot] toronto [dot] edu

Phone: 416-978-7441

Office: BA 5230

Office hours: by appointment

HANDOUTS:

Course outline

Lecture slides

ASSIGNMENTS:

Assignment 1 -- No more questions will be added

Python module bonnerlib.py

Assignment 2 -- No more questions will be added

Assignment 3 -- No more questions will be added

Data: MNIST handwritten digits, compressed (18 MB) and uncompressed (213 MB).

Assignments should be submitted electronically at the UTORSubmit website.

TEXT:

Bernhard Scholkopf and Alex Smola, Learning with Kernels, MIT Press, 2002.

About a third of the book is freely available on the book's web page (click on "Contents"), as are numerous lecture slides.

ADDITIONAL REFERENCES:

A quick review of real symmetric matrices.

Cristianini and Shawe-Taylor, An Introduction to Support Vector Machines, Cambridge University Press, 2000.

Shawe-Taylor and Cristianini, Kernel Methods for Pattern Analysis, Cambridge University Press, 2004.

Steinwart and Christmann, Support Vector Machines, Springer, 2008.

Boyd and Vandenberghe, Convex Optimization, Cambridge University Press, 2004. Freely available on the web.

Hastie, Tibshirani and Friedman, The Elements of Statistical Learning, Springer, 2001.

BACKGROUND MATERIAL:

Lipschutz and Lipson, Schaum's Outline of Linear Algebra. (very handy, very cheap)

Wrede and Spiegle, Schaum's Outline of Advanced Calculus. (very handy, very cheap)

SOFTWARE:

The Python programming language

The NumPy libraries (Numerical Python)

The SciPy libraries (Scientific Python)

The scikit-learn libraries (machine learning in Python)

The Spyder IDE (Scientific Python Development Environment) (optional)

The easiest way to install this software on your own computer is to download and install Anaconda, a Python-based data-science platform that includes many popular data-science packages, including NumPy, SciPy, scikit-learn and Spyder.

Getting started with SciPy

SciPy documentation

Tutorial on machine learning in scikit-learn

Support Vector Machines in scikit-learn

SciPy lecture notes by Valentin Haenel, Emmanuelle Gouillart, and Gael Varoquaux (eds)

PLAGIARISM AND CHEATING:

The academic regulations of the University are outlined in the Code of Behaviour on Academic Matters.

Advice on academic offences