Homepage for CSC 2545, Winter 2017
Kernel Methods and Support Vector Machines
Department of Computer Science
University of Toronto
I have added a separate submission portal on UTORSubmit for Question 5 of assignment 3. (If you have already submitted question 5 to the portal for assignment 3, you can just leave it there, but any updates should be submitted to the new portal. Any material in the new portal will be assumed to be your definitive submission for question 3.)
The deadline for Question 5 of Assignment 3 is extended until April 24 at 11pm. However, if you need your final course grade by April 21 (for June graduation), Question 5 must be handed in by April 18 at 11pm. Questions 1-4 are still due on April 16 at 11pm for everyone.
Marked Assignment 2s can be picked up from the TA (BA6135) on April 20 (not 19) from noon to 2pm. Please email him before you come and give your approximate arrival time (shuangjian [dot] zhang [at] mail [dot] utoronto [dot] ca).
The deadline for Question 5 of Assignment 3 will be extended. Details to come.
Question 4(f) on Assignment 3 has been clarified.
In Question 4 of Assignment 3, you may assume that K is symmetric and positive definite. You may also assume that equation (1) has a minimum.
Questions 4(e) and (f) on Assignment 3 have been changed. (They are now more specific and should be easier to answer.)
In Question 5 of Assignment 3, you should be able to obtain a cross-validation error of less than 0.02
The deadline for Assignment 3 has been extended to Sunday April 16 at 11pm.
Assignment 3 is now complete. No more questions will be added.
The last class will be on April 6.
The first four questions of Assignment 3 are niw available, below.
Assignment 2 is now complete. No more questions will be added.
A programming problem has been added to Assignment 2. The due date and submission instructions are described at the start of the assignment sheet.
A programming question will be added to Assignment 2, but you will have an extra week to hand it in. All other questions are due on March 23.
The last class will be on April 6.
The role of symmetry in Question 5 of Assignment 2 has been clarified. Please see the assignment sheet.
The first five questions of Assignment 2 have been posted, below.
The first three questions of Assignment 2 have been posted, below.
Assignment 2 is due on Thursday March 23.
Computer programs for all assignments should be submitted electronically at the UTORSubmit website, as described below.
All questions have now been added to Assignment 1.
The number of test points in Question 4(a) of Assignment 1 as been reduced from 20,000 to 2000.
Questions 2 and 3 in Assignment 1 have been slightly elaborated.
An introduction to Question 4 has been added to Assignment 1.
A third question has been added to Assignment 1.
As discussed in class, Question 2 of Assignment 1 should refer to Question 7.13 in the text, not Question 7.10. (See the corrected assignment, below)
The first two questions of Assignment 1 are now available, below. More questions will be added shortly.
The introduction of Support Vector Machines (SVMs) in the
1990s led to an explosion of applications and deepening
theoretical analysis that have established SVMs as one of the
standard tools for machine learning and data mining. This course
provides a comprehensive introduction to SVMs and other kernel
methods, including theory, algorithms and applications. Topics
covered will be selected from the following: support vector
classification and regression; novelty detection and feature
extraction; non-linear dimensionality reduction; reproducing
kernel maps; regularization; statistical learning theory and
robust estimation; convex optimization and implementation;
kernel design and applications. Homework assignments will be a
mix of theory and programming.
Research Area 12 (Machine Learning), Methodology 2
Lectures: Thursday 2-4pm. The first class will be on January 12. The last class will be on April 6.
Location: OI 4-414 (OISE, 252 Bloor St W)
Expected work: Three assignments, which will be a mix of
theory and programming
Prerequisites: Linear algebra, vector calculus, basic
probability, and a willingness to program in Python. A
previous course in machine learning is not required but would
be helpful. Mathematical maturity will be assumed.
email: my last name [at] cs [dot] toronto [dot] edu
Office: BA 5230
Office hours: by appointment
Assignment 1 -- No more questions will be added
Python module bonnerlib.py
Assignment 2 -- No more questions will be added
Assignment 3 -- No more questions will be added
Data: MNIST handwritten digits, compressed (18 MB) and uncompressed (213 MB).
Assignments should be submitted electronically at the UTORSubmit website.
Bernhard Scholkopf and Alex Smola, Learning with
Kernels, MIT Press, 2002.
About a third of the book is freely available on the book's
web page (click on "Contents"), as are numerous lecture slides.
A quick review of real
Cristianini and Shawe-Taylor, An Introduction to
Support Vector Machines, Cambridge University Press, 2000.
Shawe-Taylor and Cristianini, Kernel Methods for
Pattern Analysis, Cambridge University Press, 2004.
Steinwart and Christmann, Support
Vector Machines, Springer, 2008.
Boyd and Vandenberghe, Convex
Optimization, Cambridge University Press, 2004. Freely
available on the web.
Hastie, Tibshirani and Friedman, The
Elements of Statistical Learning, Springer, 2001.
Lipschutz and Lipson, Schaum's Outline of Linear Algebra.
(very handy, very cheap)
Wrede and Spiegle, Schaum's Outline of Advanced Calculus.
(very handy, very cheap)
The Python programming language
The NumPy libraries (Numerical Python)
The SciPy libraries (Scientific Python)
The scikit-learn libraries (machine learning in Python)
The Spyder IDE (Scientific Python Development Environment) (optional)
The easiest way to install this software on your own computer is to download and install Anaconda, a Python-based data-science platform that includes many popular data-science packages, including NumPy, SciPy, scikit-learn and Spyder.
Getting started with SciPy
Tutorial on machine learning in scikit-learn
Support Vector Machines in scikit-learn
SciPy lecture notes by Valentin Haenel, Emmanuelle Gouillart, and Gael Varoquaux (eds)
PLAGIARISM AND CHEATING:
The academic regulations of the University are outlined in
of Behaviour on Academic Matters.
on academic offences