CSC2535 Spring 2013 - Lectures

DRAFT LECTURE SCHEDULE: SUBJECT TO CHANGE

The final versions of the lecture notes will generally be posted on the webpage around the time of the lecture.
  • January 9
    Lecture 1: Overview of Machine Learning and Graphical Models
    notes as ppt , notes as .pdf
    Reading: Bishop, Chapter 8: pages 359-399

  • January 16
    Lecture 2a: Inference in Factor Graphs
    notes as ppt , notes as .pdf
    Lecture 2b: Variational Inference and the wake-sleep algorithm notes as ppt , notes as .pdf
    Reading: Bishop, Chapter 8: pages 399-418, 450-455
    Reading: "The wake-sleep algorithm for unsupervised Neural Networks." .pdf
    Reading: "A view of the EM algorithm that justifies incremental, sparse, and other variants." .pdf

  • January 23 (first assignment posted on web)
    Lecture 3a: The origin of variational Bayes
    notes as ppt , notes as .pdf
    Required reading: Training Products of Experts by Minimizing Contrastive Divergence. [pdf]
    Lecture 3b: Approximate Learning Methods for Energy-Based Models
    notes as ppt , notes as .pdf
    Reading: "Keeping Neural Networks Simple by Minimizing the Description Length of the Weights." .pdf
    Reading: "Training products of experts by minimizing contrastive divergence" .pdf
    Reading: "Contrastive Backpropagation" .pdf

  • January 30
    Lecture 4: Restricted Boltzmann machines
    notes as ppt , notes as .pdf
    Required reading: Training Restricted Boltzmann Machines using Approximations to the Likelihood Gradient. [pdf]
    Reading: Estimation of non-normalized statistical models using score matching. [pdf]

  • February 6: First assignment due (at start of class)
    Lecture 5: Deep Boltzmann machines
    notes as .ppt notes as .pdf
    Reading: "An efficient learning procedure for deep Boltzmann machines" .pdf

  • February 13 (second assignment posted on web)
    Lecture 6: Object Recognition in Deep Neural Nets
    notes as ppt , notes as .pdf
    notes as ppt , notes as .pdf
    Reading for lecture 6a: ImageNet Classification with Deep Convolutional Neural Networks. [pdf]
    Reading for lecture 6b: Transforming autoencoders. [pdf]
    Optional reading: A Fast Learning Algorithm for Deep Belief Nets. [pdf]

  • February 20 No Lecture (reading week)

  • Feb 27 Second assignment due (at start of class)
    Lecture 7: Models of words and documents
    notes as .pdf
    Required reading: Latent Dirichlet allocation. [html]

  • March 6
    Lecture 8a: Learning three-way interactions
    notes as .ppt , notes as .pdf
    Required reading: Learning to represent spatial transformations with factored higher-order Boltzmann machines. [pdf]
    Required reading: Factored Conditional Restricted Boltzmann Machines for Modeling Motion Style. [pdf]
    Lecture 8b: Autoencoders for Image retrieval: notes as .ppt
    notes as .pdf
    .....Required reading: [.pdf]

  • March 13
    Lecture 9: Collaborative filtering and Missing Data Problems
    notes as .pdf
    Required reading: Learning from incomplete data. [pdf]
    Recommended reading: Recommender Systems: Missing Data and Statistical Model Es timation. [pdf]

  • March 20
    Lecture 10: Recurrent neural networks
    notes as .ppt , notes as .pdf
    Required reading: [.pdf]

  • March 27
    Lecture11: Non-linear Dimensionality Reduction
    notes as .ppt , notes as .pdf
    Required Reading: ISOMAP [article on web]
    Required Reading: Local Linear Embedding [.pdf]
    Required Reading: Stochastic Neighbor Embedding [.ps]
    Required Reading: Visualizing Data using t-SNE [.pdf]
    Optional reading: A paper that relates SNE to Laplacian Eigenmaps [.pdf]
    Optional reading: Dimension Reduction: A Guided Tour. [.pdf ]

  • April 3
    Final Test (1.10pm-2.00pm in the same place as the lectures)

  • Monday April 15: Project due before midnight as .pdf sent to hinton@cs.toronto.edu