Homepage for CSC 2541, Fall 2007

Topics in Machine Learning:

Kernel Methods and Support Vector Machines

University of Toronto



YOU SHOULD CHECK THE FOLLOWING ANNOUNCEMENTS REGULARLY
ANNOUNCEMENTS:
  • You might want to check out the additional references listed below.
    Course Description:

    The introduction of Support Vector Machines (SVMs) in the 1990s lead to an explosion of applications and deepening theoretical analysis that have established SVMs as one of the standard tools for machine learning and data mining. They now deliver state-of-the-art performance in real-world applications such as text categorization, hand-written character recognition, image classification, bioinformatics, etc.

    This course provides a comprehensive introduction to SVMs and other kernel methods, including theory, algorithms and applications. Topics covered will be selected from the following: support vector classification and regression; non-linear dimensionality reduction and feature extraction; designing kernels; Bayesian kernel methods; robust estimation; convex optimization; regularization; statistical learning theory; implementation; applications.

    Basic information:

  • Course area: 3A
  • Lectures: Monday and Thursday 5-6pm.
  • Location: HS 696 (Health Sciences Building, 155 College St, south side).
  • Expected work: Three or four assignments.
  • Prerequisites: Linear algebra, vector calculus, basic probability, and a willingness to program in Matlab. Mathematical maturity will be assumed.

    Instructor:

  • Anthony Bonner
  • email: my last name [at] cs [dot] toronto [dot] edu
  • Phone: 416-978-7441
  • Office: BA 4268
  • Office hours: by appointment

    Handouts:

  • Course outline
  • Lecture slides

    Assignments:

  • Assignment 1
  • Assignment 2
  • Assignment 3

    Text:

  • Bernhard Scholkopf and Alex Smola, Learning with Kernels, MIT Press, 2002.
  • The first seven chapters are freely available at the book's web page (click on "Contents"), as are numerous lecture slides.

    Additional references:

  • Cristianini and Shawe-Taylor, An Introduction to Support Vector Machines, Cambridge University Press, 2000.
  • Shawe-Taylor and Cristianini, Kernel Methods for Pattern Analysis, Cambridge University Press, 2004.
  • Hastie, Tibshirani and Friedman, The Elements of Statistical Learning, Springer, 2001.

    Background material:

  • Kolmogorov and Fomin, Elements of the Theory of Functions and Functional Analysis, Dover, 1957. (very handy, very cheap)
  • Lipschutz and Lipson, Schaum's Outline of Linear Algebra. (very handy, very cheap)
  • Wrede and Spiegle, Schaum's Outline of Advanced Calculus. (very handy, very cheap)

    Matlab:

  • Matlab Primer.
  • Matlab Intro.
  • Prof. Christara's A Brief Introduction to MatLab.
  • Cleve Moler's Introduction to MATLAB chapter from his new textbook.
  • Here is a good site for Matlab information and tutorials.
  • Another good site for Matlab information, tutorials and software.

    Octave:

  • You may use Octave instead of Matlab for homework assignments. However, I cannot guarantee to help you if you have problems. Octave is very similar to Matlab and is freely available on the web, but the user interface is not as convenient.
  • Instructions for installing and running Octave in Windows.
  • More details on installing Octave in Windows.
  • Octave manual
  • GNU Octave Repository
  • Octave Wiki

    Plagiarism and Cheating:

  • The academic regulations of the University are outlined in the Code of Behaviour on Academic Matters.
  • Advice on academic offences