CSC411F: Machine Learning and Data Mining
Fall, 2005
Tutorials
Information regarding the tutorials for CSC411 will be posted here. Please check this page regularly for announcements, notes and files.
Announcements:
I will be holding office hours on Tuesday, November 1st, between 11am and 1pm in BA4290 (Bahen Center, 4th floor) on the St. George campus, to answer questions about the midterm. I will also be returning your graded Assignment 2 during the office hours.
Tutorial 1: Introduction to Matlab
An Introduction to Matlab demo is contained in this file. Copy the file, as well as (data, poly.m) to your directory, start Matlab, and then run the demo by typing matlab_intro at the prompt.
Also check out the Matlab Resources page for tutorials and references for Matlab.
Tutorial 2: Naive Bayes and k-Nearest Neighbours
An example of Naive Bayes and kNN was shown. For details on the Naive Bayes algorithm, see the handout. Please take a look at a Matlab demo of k-NN. Refer to the instructions for running the demo. Notice how the decision boundaries change as k is varied.
Tutorial 3: Decision Tree Learning
Discussion of Entropy, Information Gain and Decision Tree Learning. Please take a look at a detailed example of decision tree learning. Ch. 2 of David MacKay's book provides a good introduction to Entropy. A link to the book is given in the Useful Links page.
Tutorial 4: Neural Networks
Discussion of perceptrons and multilayer perceptrons. Perceptron training using Gradient Descent was discussed.
Tutorial 5: Neural Networks II
Discussion of the logistic activation function. The derivation of the Backpropagation algorithm was shown. You may want to look up the Netlab toolbox for Neural Networks in the Useful Links page.
Tutorial 6: Ensemble Methods
Discussion of Ensemble methods: Boosting, Bagging and Mixture of Experts. We looked at the "Ensemble Methods in Machine Learning" paper (follow the Supplementary Reading link from course website). The "Adaptive Mixtures of Local Experts" paper mentioned in the tutorial can be found here. It is an optional supplementary reading.
Tutorial 7: K-means and Mixtures of Gaussians
Continuation of Ensemble methods: Discussion of how Bayesian methods avoid overfitting. We looked at K-means (hard and soft responsibilities), and Mixtures of Gaussians.
Tutorial 8: Midterm
Tutorial 9:
Examples of K-means Clustering, Mixture of Gaussians and PCA were shown.
Tutorial 10: Dimensionality Reduction
Discussion of ICA and Isomap. Refer to the paper in the supplementary readings.
Tutorial 11: Reinforcement Learning.
Discussion of Q Learning and TD Learning.
Tutorial 12:
Discussion of Hidden Markov Models. Refer to the relevant paper from the supplementary readings.