Neural Computation & Adaptive Perception Summer School
August 15-19, 2006
Toronto, Canada
This summer school is organized as part of the
Neural Computation & Adaptive Perception research program of CIAR.
It is hosted by the Machine Learning group at the Department of Computer Science, University of Toronto.
Tutorials
- August 15: Geoffrey Hinton, Tutorial on learning feature hierarchies
Lecture 1a: Sigmoid Belief Nets and Boltzmann Machines [.htm], [.ppt], [.ps, 4 per page]
Lecture 1b: Contrastive divergence and deterministic energy-based models [.htm], [.ppt], [.ps, 4 per page]
Related papers:
- Hinton, G. E., Osindero, S., Welling, M. and Teh, Y. Unsupervised Discovery of Non-linear Structure using Contrastive Backpropagation. To appear in Cognitive Science, 30:4, 2006. [ps] [pdf]
- Osindero, S., Welling, M. and Hinton, G. E. Topographic Product Models Applied To Natural Scene Statistics. Neural Computation, 18, pp 381-344. [pdf]
Lecture 2a: Learning deep belief nets [.htm], [.ppt], [.ps, 4 per page]
Related paper:
Lecture 2b: Autoencoders & Modeling time series with Boltzmann machines [.htm], [.ppt], [.ps, 4 per page]
Related paper:
- August 16: Yann LeCun, Tutorial on energy-based learning for vision
Lecture 1: A Tutorial on Energy-Based Learning [Slides (pdf)]
Lecture 2: Deep Learning for Generic Object Recognition [Slides (pdf)]
Related paper:
- LeCun, Y., Chopra, S., Hadsell, R., Ranzato, M., and Huang, F. J. "A Tutorial on Energy-Based Learning." To appear in "Predicting Structured Data", G. Bakir, T. Hofman, B. Scholkopf, A. Smola, B. Taskar (eds), MIT Press, 2006. [pdf]
- August 17: Sam Roweis, Tutorial on probabilistic models
Lecture 1a: Probabilistic graphical models [Slides (pdf)]
Lecture 1b: Parameter learning in fully observed graphical models [Slides (pdf)]
Lecture 2a: Latent variable models and learning with the EM algorithm [Slides (pdf)]
Lecture 2b: Inference in chains and trees, message passing, belief propagation [Slides (pdf)]
Generalized "Factoring" [Slides (pdf)]
- August 18: Kevin Murphy, Tutorial on structure learning
Graphical model structure learning [Slides (pdf)]
- August 19: Yair Weiss, Tutorial on inference and perception
Lightness Perception and Lightness Illusions.
Moving Rhombus Displays.
Talks by students
- Jack Culpepper, "Factoring movies into 'what' and 'where'" [Slides (pdf)]
- Anat Levin, "Learning to Combine Bottom-up and Top-down Segmentation" [Slides (ppt)]
Related paper:
- Levin, A. and Weiss, Y. "Learning to Combine Bottom-Up and Top-Down Segmentation." Proc. of the European Conference on Computer Vision (ECCV), Graz, Austria, May 2006. [pdf]
- Vikranth Rao, "Neural Basis of Perceptual Learning" [Slides (ppt)]
- Pierre Garrigues, "Hierarchical Sparse Bayesian Learning" [Slides (pdf)]
- Roland Memisevic, "Unsupervised learning of image transformations" [Slides (pdf)]
- Ruslan Salakhutdinov, "Nonlinear Dimensionality Reduction Using Neural Networks" [Slides (pdf)]
- Marc'Aurelio Ranzato, "Efficient Learning of Sparse Overcomplete Representations with an Energy-Based Model" [Slides (pdf)]
- Eddie Ng, "Super-Patches: Image Tesselation with Arbitrarily Shaped Patches" [Slides (ppt)]
- Frank Wood, "Gentle Introduction to Infinite Gaussian Mixture Modeling" [Slides (ppt)]
Related papers:
- Wood, F., Goldwater, S., and Black, M. J. "A non-parametric bayesian approach to spike sorting." In IEEE Engineering Medicine Biologicial Systems. Submitted, 2006. [pdf]
- Griffiths, T. L. and Ghahramani, Z. "Infinite latent feature models and the Indian buffet process." Gatsby Computational Neuroscience Unit Technical Report GCNU TR 2005-001 (2005). [pdf]
- Rasmussen, C. E. "The Infinite Gaussian Mixture Model." Advances in Neural Information Processing Systems 12, 554-560. (Eds.) Solla, S. A., T. K. Leen and K. R. Muller, MIT Press (2000). [pdf]
- Hugo Larochelle, "Generalizing to a zero-data task: a linear model case study" [Slides (pdf)]
- Leonid Sigal, "Predicting 3D People from 2D Pictures" [Slides (pdf)]
Related papers:
- Sigal, L. and Black, M. J. "Predicting 3D People from 2D Pictures." In IV Conference on Articulated Motion and Deformable Objects (AMDO), 2006. [pdf]
- Sigal, L. and Black, M. J. "Measure Locally, Reason Globally: Occlusion-sensitive Articulated Pose." IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2006. [pdf]
- Nicolas Le Roux, "Theoretical Remarks on Deep Belief Networks" [Slides (pdf)]
- Stefan Roth, "Efficient Belief Propagation for MRFs in Low-Level Vision" [Slides (pdf)]
Related papers:
- Lan, X., Roth, S., Huttenlocher, D. P., and Black, M. J. "Efficient belief propagation with learned higher-order Markov random fields." In A. Leonardis, H. Bischof, and A. Prinz, eds., Proc. of the European Conference on Computer Vision (ECCV), Volume 2, LNCS 3952, pp. 269-282, Springer Verlag, 2006. [pdf]
- Pal, C., Sutton, C., and McCallum, A. "Sparse Forward-Backward using Minimum Divergence Beams for Fast Training of Conditional Random Fields." In Proc. IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), vol. 5, pp. 581-584, 2006. [pdf]
- Mark Schmidt, "Accelerated Training of Conditional Random Fields with Stochastic Gradient Methods" [Slides (pdf)]
Related paper:
- Vishwanathan, S. V. N., Schraudolph, N. N., Schmidt, M. W., and Murphy, K. P. "Accelerated Training of Conditional Random Fields with Stochastic Gradient Methods." In International Conference on Machine Learning (ICML), 2006. [pdf]
- Scott Leishman, "Cryptogram Decoding for Optical Character Recognition" [Slides (pdf)]
Related papers:
- Laven, K., Leishman, S., and Roweis, S. "A Statistical Learning Approach To Document Image Analysis." 8th International Conference on Document Analysis and Recognition (ICDAR), pp 357-361, 2005. [pdf]
- Zhuang, L., Zhou, F., and Tygar, J. D. "Keyboard acoustic emanations revisited." Proceedings of the 12th ACM conference on Computer and communications security, pp 373-382, 2005. [keyboard-emanations.org]
- Gregory Shakhnarovich, "Metric Embedding of Task-Specific Similarity" [Slides (pdf)],
[Related work: Greg's thesis]
- Marcus Brubaker, "Dynamical Models for People Tracking" [Slides (pdf)]