Note: The test on December 8 at 3pm will be held in BA B024, not the usual lecture/tutorial room.
Radford Neal, Office: SS6016A, Phone: (416) 978-4970, Email: firstname.lastname@example.org, Office hours: Tuesdays 2:10-3:00.
Mondays and Wednesdays, 3:10pm to 4:00pm, from September 11 to December 6, except for Thanksgiving (October 9). Held in BA1220.
Fridays, 3:10pm to 4:00pm, from September 22 to December 8. Held in BA1220.
TA: Renqiang Min
NOTE: Tests will be held during the tutorial time, so you must not have a time conflict with the tutorial.
Three in-class tests, worth 20% each. Tentatively scheduled for October 13, November 17, and December 8.
Four assignments, worth 10% each. Tentatively due on October 11, November 1, November 15, and November 29 (handed out two weeks earlier).
There is no textbook for this course. Lecture slides will be available from this web page. I will also post links to various on-line references.
The assignments will involve writing programs in Matlab/Octave or R. You can download Octave and R for your home computer for free from their web sites. You can also obtain an account on the CS department's CDF computer system, where R and Matlab are available - just type "R" or "matlab" (or "matlab -nosplash -nojvm" for a less GUI version).
The slides are available in Postscript (ps) and Portable Document Format (pdf), in both one-per-page and four-per-page (4up) versions.
Introduction: ps, pdf, ps-4up, pdf-4up
Nearest-neighbor and linear regression: ps, pdf, ps-4up, pdf-4up
Probability and loss functions: ps, pdf, ps-4up, pdf-4up
Naive Bayes classifiers: ps, pdf, ps-4up, pdf-4up
Logistic regression (revised and extended): ps, pdf, ps-4up, pdf-4up
Decision trees (revised and extended): ps, pdf, ps-4up, pdf-4up
Cross validation: ps, pdf, ps-4up, pdf-4up
Neural networks: ps, pdf, ps-4up, pdf-4up
Maximum likelihood estimation for neural networks: ps, pdf, ps-4up, pdf-4up
Training neural networks with early stopping: ps, pdf, ps-4up, pdf-4up
Bayesian learning: ps, pdf, ps-4up, pdf-4up
Bayesian neural networks: ps, pdf, ps-4up, pdf-4up
Markov chain Monte Carlo: ps, pdf, ps-4up, pdf-4up
Clustering: ps, pdf, ps-4up, pdf-4up
Mixture models (revised and extended): ps, pdf, ps-4up, pdf-4up
Principal Component Analysis (revised): ps, pdf, ps-4up, pdf-4up
Factor Analysis (extended): ps, pdf, ps-4up, pdf-4up
Nonlinear dimensionality reduction: ps, pdf, ps-4up, pdf-4up
Reinforcement learning: ps, pdf, ps-4up, pdf-4up
Q learning (revised and extended): ps, pdf, ps-4up, pdf-4up
Assignment 1:Handout: ps, pdf.Assignment 2:
Due date has been extended to October 18.
Data set 1: train x, train y, test x, test y.
Data set 2: train x, train y, test x, test y.
Maximum likelihood logistic regression in R: all functions.
Maximum likelihood logistic regression in Matlab: estimation, likelihood, prediction.
Fudged functions needed for the old Matlab on CDF: fudged estimation, fudged minus log likeihood.
Solution: R functions for ordinary LR, R functions for bounded LR, R test script, output of script, plots produced by script.Handout: ps, pdf.Assignment 3:
Data files: train x, train y, test x, test y.
Solution: MLP functions, script, output, plots.Handout: ps, pdf.Assignment 4:
Here are some notes on how to do some things you'll need to do.
Here is the data. Note: I mistakenly put only 49 cases in the data file. That's OK. Just use 49 rather than 50.
Solution: Main functions, script, output plots.Handout: ps, pdf.
Here is the gene expression data and the cancer indicators.
Demo of K-NN and linear regression in R using prostate cancer data: knn program, script, training data, test data.
Demo of Naive Bayes in Matlab/Octave: nbayes program, script, training inputs, training targets, test inputs, test targets.
Demo of decision trees using the R "rpart" function: R script, output of script.
K means function and example: R function for K means, script to try it out.
Q learning program and demo: Q learning functions, Demo 1, Demo 2, Demo 2m.
These programs are also available on CDF in /u/radford/411/demo.
Some useful on-line references
Proceedings of the annual conference on Neural Information Processing Systems (NIPS)
Information Theory, Inference, and Learning Algorithms, by David MacKay
Reinforcement Learning: An Introduction, by Richard S. Sutton and Andrew G. Barto
My tutorial on Bayesian methods for machine learning: Postscript or PDF.
Web pages for past related courses:
STA 414 (Spring 2006)
CSC 321 (Spring 2006, Geoffrey Hinton)
CSC 411 (Fall 2005, Anthony Bonner)
CSC 411 (Fall 2004, Richard Zemel)