About Me
I am a postdoctoral researcher in the machine learning group at
the University of Toronto. My research interests are primarily in
machine learning with a strong interest in Bayesian statistics. I
have been involved in numerous projects that involve the
application of machine learning to problems in health informatics.
I was formerly a member of
the
Intelligent
Assistive Technology and Systems Lab.
News & Things on the radar
In September 2013, I will be starting a postdoctoral fellowship at Harvard University in the School of Engineering and Applied Sciences.
Bayesian Optimization code now up in the software section.
Practical Bayesian Optimization of Machine Learning Algorithms will appear at NIPS.
New paper in Journal of Machine Learning research.
April 21-23, 2012: Attending AISTATS conference in La Palma.
Selected Publications
NIPS 2012
Practical Bayesian Optimization of Machine Learning Algorithms
Jasper Snoek, Hugo Larochelle and Ryan Prescott Adams.
Advances in Neural Information Processing Systems, 2012
The use of machine learning algorithms frequently involves careful
tuning of learning parameters and model
hyperparameters. Unfortunately, this tuning is often a “black art”
requiring expert experience, rules of thumb, or sometimes brute-force
search. There is therefore great appeal for automatic approaches that
can optimize the performance of any given learning algorithm to the
problem at hand. In this work, we consider this problem through the
framework of Bayesian optimization, in which a learning algorithm’s
generalization performance is modeled as a sample from a Gaussian
process (GP). We show that certain choices for the nature of the GP,
such as the type of kernel and the treatment of its hyperparameters,
can play a crucial role in obtaining a good optimizer that can achieve
expert- level performance. We describe new algorithms that take into
account the variable cost (duration) of learning algorithm experiments
and that can leverage the presence of multiple cores for parallel
experimentation. We show that these proposed algorithms improve on
previous automatic procedures and can reach or surpass human
expert-level optimization for many algorithms including latent
Dirichlet allocation, structured SVMs and convolutional neural
networks.
@inproceedings{snoek-etal-2012b,
author = {Jasper Snoek and Hugo Larochelle and Ryan Prescott Adams},
title = {Practical Bayesian Optimization of Machine Learning Algorithms},
year = {2012},
booktitle = "Neural Information Processing Systems"
}
JMLR 2012
Nonparametric Guidance of Autoencoder Representations using Label Information
Jasper Snoek, Ryan Prescott Adams, Hugo Larochelle.
Journal of Machine Learning Research, 2012
While unsupervised learning has long been useful for density modeling,
exploratory data analysis and visualization, it has become
increasingly important for discovering features that will later be
used for discriminative tasks. Discriminative algorithms often work
best with highly-informative features; remarkably, such features can
often be learned without the labels. One particularly effective way to
perform such unsupervised learning has been to use autoencoder neural
networks, which find latent representations that are constrained but
nevertheless informative for reconstruction. However, pure
unsupervised learning with autoencoders can find representations that
may or may not be useful for the ultimate discriminative task. It is a
continuing challenge to guide the training of an autoencoder so that
it finds features which will be useful for predicting
labels. Similarly, we often have a priori information regarding what
statistical variation will be irrelevant to the ultimate
discriminative task, and we would like to be able to use this for
guidance as well. Although a typical strategy would be to include a
parametric discriminative model as part of the autoencoder training,
here we propose a nonparametric approach that uses a Gaussian process
to guide the representation. By using a nonparametric model, we can
ensure that a useful discriminative function exists for a given set of
features, without explicitly instantiating it. We demonstrate the
superiority of this guidance mechanism on four data sets, including a
real-world application to rehabilitation research. We also show how
our proposed approach can learn to explicitly ignore statistically
significant covariate information that is label-irrelevant, by
evaluating on the small NORB image recognition problem in which pose
and lighting labels are available.
@article {snoek-etal-2012c,
title = {Nonparametric Guidance of Autoencoder Representations
using Label Information},
journal = {Journal of Machine Learning Research},
volume = {13},
year = {2012},
month = {10/2012},
pages = {2567-2588},
author = {Jasper Snoek and Ryan Prescott Adams and Hugo Larochelle}
}