Notes

NB: Until I get around to making progress on my barely started book, these are the best I've got to offer. But some are quite old, and none have been carefully checked over, so use at your own risk.

Also, I would be remiss if I didn't point you to

• Kevin Murphy, who has many excellent tutorials and research notes on his page
• Thomas Minka who is seemingly unstoppable and will fill you in on matrix calculus all things Bayesian
• Adam Berger who has some good introductory notes on MaxEnt modeling
• Other pages listing matrix identities:
  [ Imperial, Resa, CMU, Utah]

• Useful Matrix and Gaussian formulae
  Two "cheat-sheets" of useful matrix and Gaussian formulae for anyone who needs to take inverses or derivatives or conditional expectations using linear operators and normal densities. (Recently fixed an often reported error in eq 4c; in equation 3e the X^{-1} terms on the right should be transposed.)
  Matrix Identities     [revised june 99-->ps.gz(27K) pdf(122K)]
  Gaussian Identities [revised july 99-->ps.gz(24K) pdf(90K)]
  

• Speech processing background
  A short tutorial on speech processing methods and terminology for those getting started in recognition, speaker id, synthesis, etc. (this became Appendix A in my thesis).
  [version of nov.98-->ps.gz (letter)(313K) ps.gz (A4)(313K)]
  

Tutorials

• Graphical Models, HMMs, Linear Dynamical Systems, Unsupervised Learning, SVMs and VC Dimension
• See my tutorials page for the notes

Probably Useless Notes

• Finding eigenvectors in high dimensions
  A short review of the snap-shot method for finding the first few eigenvectors of a small dataset in a very high dimensional space.
  [version of dec.96-->ps.gz(40K) pdf(123K)]
  

• Levenberg-Marquardt optimization
  A brief note on this optimization method which is a mix of vanilla gradient descent and second order curvature methods.
  [version of nov.96-->ps.gz(39K) pdf(123K)]
  

• Hidden Markov models
  A brief note on these simple factor-analysis-through-time models.
  [version of feb.97-->ps.gz(41K) pdf(128K)]
  

• Boltzmann machines
  An old tutorial note on these early symmetric belief-type networks.
  [version of nov.95-->ps.gz(51K) pdf(156K)]
  

Old Lecture Slides

• Unsupervised learning
  Slides from a lecture I gave on unsupervised networks.
  [version of nov.96-->ps.gz(88K)]
  

• Boltzmann machines & Stochasic Networks
  Slides from a lecture I gave on these undirected networks.
  [version of nov.96-->ps.gz(65K)]
  

• The EM algorithm
  Slides from a lecture introducing this parameter estimation technique.
  [version of nov.96-->ps.gz(62K)]
  

• Bits back
  Slides (bad ones) from the first group meeting I gave at Caltech, on this idea that Geoff Hinton taught me in Toronto.
  [version of feb.95-->ps.gz(64K)]
  

Sam Roweis, Machine Learning Group, Toronto Computer Science, www.cs.toronto.edu/~roweis