Toronto, Canada

This summer school is organized as part of the
Neural Computation & Adaptive Perception research program of CIAR.

It is hosted by the Machine Learning group at the Department of Computer Science, University of Toronto.

- August 15: Geoffrey Hinton, Tutorial on learning feature hierarchies Lecture 1a: Sigmoid Belief Nets and Boltzmann Machines [.htm], [.ppt], [.ps, 4 per page]
- Hinton, G. E., Osindero, S., Welling, M. and Teh, Y. Unsupervised Discovery of Non-linear Structure using Contrastive Backpropagation. To appear in Cognitive Science, 30:4, 2006. [ps] [pdf]
- Osindero, S., Welling, M. and Hinton, G. E. Topographic Product Models Applied To Natural Scene Statistics. Neural Computation, 18, pp 381-344. [pdf]
- Hinton, G. E., Osindero, S. and Teh, Y. A fast learning algorithm for deep belief nets. (A revised version will appear in Neural Computation.)
[ps.gz]
[pdf]

Movies of the neural network generating and recognizing digits.

- Hinton, G. E. and Salakhutdinov, R. R. Reducing the dimensionality of data with neural networks. Science, Vol. 313. no. 5786, pp. 504 - 507, 28 July 2006.

[full paper, Matlab code, and supporting online material] - August 16: Yann LeCun, Tutorial on energy-based learning for vision Lecture 1: A Tutorial on Energy-Based Learning [Slides (pdf)]
- LeCun, Y., Chopra, S., Hadsell, R., Ranzato, M., and Huang, F. J. "A Tutorial on Energy-Based Learning." To appear in "Predicting Structured Data", G. Bakir, T. Hofman, B. Scholkopf, A. Smola, B. Taskar (eds), MIT Press, 2006. [pdf]
- August 17: Sam Roweis, Tutorial on probabilistic models Lecture 1a: Probabilistic graphical models [Slides (pdf)]
- August 18: Kevin Murphy, Tutorial on structure learning Graphical model structure learning [Slides (pdf)]
- August 19: Yair Weiss, Tutorial on inference and perception Lightness Perception and Lightness Illusions.

Lecture 1b: Contrastive divergence and deterministic energy-based models [.htm], [.ppt], [.ps, 4 per page]

Related papers:

Lecture 2a: Learning deep belief nets [.htm], [.ppt], [.ps, 4 per page]

Related paper:

Lecture 2b: Autoencoders & Modeling time series with Boltzmann machines [.htm], [.ppt], [.ps, 4 per page]

Related paper:

Lecture 2: Deep Learning for Generic Object Recognition [Slides (pdf)]

Related paper:

Lecture 1b: Parameter learning in fully observed graphical models [Slides (pdf)]

Lecture 2a: Latent variable models and learning with the EM algorithm [Slides (pdf)]

Lecture 2b: Inference in chains and trees, message passing, belief propagation [Slides (pdf)]

Generalized "Factoring" [Slides (pdf)]

Moving Rhombus Displays.

- Jack Culpepper, "Factoring movies into 'what' and 'where'" [Slides (pdf)]

- Anat Levin, "Learning to Combine Bottom-up and Top-down Segmentation" [Slides (ppt)]

Related paper:

- Levin, A. and Weiss, Y. "Learning to Combine Bottom-Up and Top-Down Segmentation." Proc. of the European Conference on Computer Vision (ECCV), Graz, Austria, May 2006. [pdf]

- Levin, A. and Weiss, Y. "Learning to Combine Bottom-Up and Top-Down Segmentation." Proc. of the European Conference on Computer Vision (ECCV), Graz, Austria, May 2006. [pdf]
- Vikranth Rao, "Neural Basis of Perceptual Learning" [Slides (ppt)]

- Pierre Garrigues, "Hierarchical Sparse Bayesian Learning" [Slides (pdf)]

- Roland Memisevic, "Unsupervised learning of image transformations" [Slides (pdf)]

- Ruslan Salakhutdinov, "Nonlinear Dimensionality Reduction Using Neural Networks" [Slides (pdf)]

- Marc'Aurelio Ranzato, "Efficient Learning of Sparse Overcomplete Representations with an Energy-Based Model" [Slides (pdf)]

- Eddie Ng, "Super-Patches: Image Tesselation with Arbitrarily Shaped Patches" [Slides (ppt)]

- Frank Wood, "Gentle Introduction to Infinite Gaussian Mixture Modeling" [Slides (ppt)]

Related papers:

- Wood, F., Goldwater, S., and Black, M. J. "A non-parametric bayesian approach to spike sorting." In IEEE Engineering Medicine Biologicial Systems. Submitted, 2006. [pdf]

- Griffiths, T. L. and Ghahramani, Z. "Infinite latent feature models and the Indian buffet process." Gatsby Computational Neuroscience Unit Technical Report GCNU TR 2005-001 (2005). [pdf]

- Rasmussen, C. E. "The Infinite Gaussian Mixture Model." Advances in Neural Information Processing Systems 12, 554-560. (Eds.) Solla, S. A., T. K. Leen and K. R. Muller, MIT Press (2000). [pdf]

- Wood, F., Goldwater, S., and Black, M. J. "A non-parametric bayesian approach to spike sorting." In IEEE Engineering Medicine Biologicial Systems. Submitted, 2006. [pdf]
- Hugo Larochelle, "Generalizing to a zero-data task: a linear model case study" [Slides (pdf)]

- Leonid Sigal, "Predicting 3D People from 2D Pictures" [Slides (pdf)]

Related papers:

- Sigal, L. and Black, M. J. "Predicting 3D People from 2D Pictures." In IV Conference on Articulated Motion and Deformable Objects (AMDO), 2006. [pdf]

- Sigal, L. and Black, M. J. "Measure Locally, Reason Globally: Occlusion-sensitive Articulated Pose." IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2006. [pdf]

- Sigal, L. and Black, M. J. "Predicting 3D People from 2D Pictures." In IV Conference on Articulated Motion and Deformable Objects (AMDO), 2006. [pdf]
- Nicolas Le Roux, "Theoretical Remarks on Deep Belief Networks" [Slides (pdf)]

- Stefan Roth, "Efficient Belief Propagation for MRFs in Low-Level Vision" [Slides (pdf)]

Related papers:

- Lan, X., Roth, S., Huttenlocher, D. P., and Black, M. J. "Efficient belief propagation with learned higher-order Markov random fields." In A. Leonardis, H. Bischof, and A. Prinz, eds., Proc. of the European Conference on Computer Vision (ECCV), Volume 2, LNCS 3952, pp. 269-282, Springer Verlag, 2006. [pdf]

- Pal, C., Sutton, C., and McCallum, A. "Sparse Forward-Backward using Minimum Divergence Beams for Fast Training of Conditional Random Fields." In Proc. IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), vol. 5, pp. 581-584, 2006. [pdf]

- Lan, X., Roth, S., Huttenlocher, D. P., and Black, M. J. "Efficient belief propagation with learned higher-order Markov random fields." In A. Leonardis, H. Bischof, and A. Prinz, eds., Proc. of the European Conference on Computer Vision (ECCV), Volume 2, LNCS 3952, pp. 269-282, Springer Verlag, 2006. [pdf]
- Mark Schmidt, "Accelerated Training of Conditional Random Fields with Stochastic Gradient Methods" [Slides (pdf)]

Related paper:

- Vishwanathan, S. V. N., Schraudolph, N. N., Schmidt, M. W., and Murphy, K. P. "Accelerated Training of Conditional Random Fields with Stochastic Gradient Methods." In International Conference on Machine Learning (ICML), 2006. [pdf]

- Vishwanathan, S. V. N., Schraudolph, N. N., Schmidt, M. W., and Murphy, K. P. "Accelerated Training of Conditional Random Fields with Stochastic Gradient Methods." In International Conference on Machine Learning (ICML), 2006. [pdf]
- Scott Leishman, "Cryptogram Decoding for Optical Character Recognition" [Slides (pdf)]

Related papers:

- Laven, K., Leishman, S., and Roweis, S. "A Statistical Learning Approach To Document Image Analysis." 8th International Conference on Document Analysis and Recognition (ICDAR), pp 357-361, 2005. [pdf]

- Zhuang, L., Zhou, F., and Tygar, J. D. "Keyboard acoustic emanations revisited." Proceedings of the 12th ACM conference on Computer and communications security, pp 373-382, 2005. [keyboard-emanations.org]

- Laven, K., Leishman, S., and Roweis, S. "A Statistical Learning Approach To Document Image Analysis." 8th International Conference on Document Analysis and Recognition (ICDAR), pp 357-361, 2005. [pdf]
- Gregory Shakhnarovich, "Metric Embedding of Task-Specific Similarity" [Slides (pdf)],
[Related work: Greg's thesis]

- Marcus Brubaker, "Dynamical Models for People Tracking" [Slides (pdf)]