CSC2515 - Introduction to Machine Learning
Fall 2005

Manage Weekly Readings

Student Number: Reading: List all submitted readings Week 1: Valiant, A Theory of the Learnable Week 2: Fisher, The Use of Multiple Measurements in Taxonomic Problems Week 3: Domingos and Pazzani, On the Optimality of the Simple Bayesian Classifier Week 4: Jordan, Why the logistic function? Week 5: Rumelhart et al., Learning representation by backpropagating errors Week 6: Jordan and Jacobs, Hierarchical Mixtures of Experts and the EM Algorithm Week 7: Chow and Liu, Approximating discrete probability distributions with dependence trees Week 8: Hinton and Neal, A View of the EM Algorithm Week 9: Ghahramani and Hinton, The EM algorithm for Mixtures of Factor Analyzers Week 10: Poritz, Hidden Markov Models: A guided tour. Week 11: Saul and Pereira, Aggregate and mixed-order Markov models for statistical language processing Week 12: Shapire, The Strength of Weak Learnability Week 13: Cortes and Vapnik, Support Vector Networks

Final Project Submission Confirmation:

If you sent in your project before 9:50am on Monday, December 19th you should be on this list. If your submission was received after 9:00am it will be considered late. We will inform you of the penalties when we decide on a discount scheme. Abida Raouf Arnold Binas Christian Lessig Clement Chung Dan Vogel Derek Nowrouzezahrai Dimitris Tsirogiannis Eddie K. H. Ng Elizabeth Lam Eugene Vahlis Fan, Jing W. Frank Chu Gerald Quon Gokul Soundararajan Gordon Cook Hao (Victor) Ren Hassan Goldani Moghaddam Hui Lan Ian Vollick Igor Naverniouk Ilya Sutskever Inmar Givoni Jacobo Bibliowicz Jake Klamka Jasper Snoek Jeff Kang Jerry Lam Jifang Kai Yi Po Keir Mierle Kelvin Chan Li Yan Manos Papagelis Martin de Lasa Mathe Stefan Milind Joshi Nikos Sarkas Oktie Hassanzadeh Peter Junteng Liu Scott Leishman Sebastian Mieth Stephen Fung Tijmen Tieleman Todd Goodwin Thomas Diamantis

Example Final Project

To make sure everyone is clear on what we expect for the projects, I've put together an example final project based on the project I submitted for the course a few years ago. Don't worry about the actual topic of the example paper, but do take a look at how the sections are laid out, what information is presented in each, and at what level of detail. If the main body of your paper doesn't roughly follow the same organization as the example project (methods/models, data sets, experimental protocol, results) it is probably lacking in machine learning content. The example project does a good job of meeting all the minimum requirements set out in the project info sheet available on the assignments page. The example project would earn a low A grade, and note that it is only 6 pages long. A higher score could be earned, while staying within the page limit, by exceeding one or more of the requirements. For example, including a more detailed analysis of the results, running an additional type of experiment, comparing performance against an additional method, etc... Example project (PS) Example project (PDF)

Assignment 2: Regression

[06/11/2005] Logsum code: Here's the code for the logsum trick. It will be covered later in the course so you should only use it if you're sure you understand how and why it works. -Ben [01/11/2005] Plotting Code: The matlab plotting package for assignment 2 is available here: a2plotting.zip. The functions are all documented in the code. Look at try_me.m for an example of how to use the functions. Please look over the example and comments in the code before e-mailing any questions. -Ben [29/10/2005] Sanity Check: A few people have asked about the leave-one-out cross validation error for polynomial and radial basis regression. We will look into simplifying the experimental protocol. In the mean time, you should focus on getting the individual regression methods working. To help verify that your implementation is correct, we're providing data sets satisfying the assumptions of each of the methods. The data is available in matlab and ascii text formats. To verify your code you can run the following experiments: polynomial_1D data: 1D Polynomial regression with K=2 polynomial_2D data: 2D Polynomial regression with K=2 radial_basis_1D data: 1D Radial basis regression with K=2 radial_basis_2D data: 2D Radial basis regression with K=2 local_1D data: 1D Locally weighted regression local_2D data: 2D Locally weighted regression A correctly implemented polynomial regression method should get average squared test errors of about 0.001 on the 1D and 2D polynomial data. A correctly implemented radial basis regression method should get average squated test errors of about 0.001 on the 1D and 2D radial basis data. A correctly implemented locally weighted regression method should get average squared test errors between 0.006 and 0.001 on the 1D and 2D radial basis data. You are not required to hand in any results on this data, but it is a convenient way to check that you're on the right track. -Ben   [25/10/2005] Corrections: 1) There was a small typo in the intro to question 1 which is now fixed online, for the constant model K=0 and h_0=x1^0x2^0=1; for the linear model K=1 but h_0 is the same: h_0=x1^0x2^0=1. 2) Also, in the third part of question 1, assume the distribution p() is UNIFORM with range [0,2mu] and mean mu. Remember that if x is uniformly distributed on [0,2mu], the expected value of x^n is (2mu)^n / (n+1). Remember as well that if a set of x_i's are independent, the expectation E[ (\prod_i x_i^a_i ) ^2 ] = E[ \prod_i x_i^{2a_i} ] = \prod_i E [x_i^{2a_i}]. That pretty much gives away the whole question, but at least it should be clear now! -Sam

Tutorial 3: Assignment 2 Questions

Sanity check data (matlab, ascii text) Change to experimental protocol (see updated assignment sheet Verifying gradients by calculating finite differences (see a random link on finite differences) Overview of the plotting package a2plotting.zip

Tutorial 1: Prob and Stats Review

Review of probability and statistics: Notes. Understanding Valiant's paper.

This site last updated Oct 3, 2008. © Benjamin Marlin