Max Welling
Learning to Herd and Herding to Learn
Learning in the traditional sense focusses on finding a point
estimate for the
parameters of its model. Bayesian approaches extend this to posterior
distributions
but are computationally intractable for Markov random fields and
inference can easily get trapped
in local modes. A third possiblility is to define a dynamical system,
herding, that mixes very
efficiently over an attractor set and where each point on this set
defines an "energy function"
over some state space. The collection of all these energy minima
represent a sample collection
that shares certain moment statistics with the input data.
I will briefly introduce this system and present very preliminary
ideas on the following issues:
- Can we learn (hyper)parameters for herding so that it can be run with the data decoupled from it?
- Which herding systems should be considered equivalent and what are the implications for the use of fast weights in ML learning?
- Among all non-equivalent herding systems that reproduce the same moment constraints, can we learn the one that performs optimal in terms of generalization?
- Can we characterize the attractor set of herding and is its dynamics chaotic?
- How useful can herding be for producing deep representations?
Brief Bio.
Max Welling is associate professor at UC Irvine with a joint appointment in the statistics department. He is associate director of the Center for Machine Learning and Intelligent Systems, associate editor for TPAMI and JCGS journals. He received multiple grants from NSF and ONR-MURI among which an NSF career grant in 2005. He was recipient of the Dean's midcareer award for research in 2008. He was conference chair for the Conference on AI and Statistics in 2009. Before joining UCI he held postdoctoral positions at Caltech ('98-'00), University College London ('00-'01) and the University of Toronto ('01-'03). He received his PhD in '98 in theoretical physics.