A Hierarchical Community of Experts
Geoffrey E. Hinton, Brian Sallans and Zoubin Ghahramani
Department of Computer Science
University of Toronto
Ontario, Canada
Abstract
We describe a directed acyclic graphical model that contains a
hierarchy of linear units and a mechanism for dynamically selecting an appropriate subset
of these units to model each observation. The non-linear selection mechanism is a
hierarchy of binary units each of which gates the output of one of the linear units.
There are no connections from linear units to binary units, so the generative model
can be viewed as a logistic belief net (Neal 1992) which selects a skeleton linear model
from among the available linear units. We show that Gibbs sampling can be used to
learn the parameters of the linear and binary units even when the sampling is so brief
that the Markov chain is far from equilibrium.
Learning in Graphical Model, 479-494, Kluwer Academic
Publishers
Download: [ps.gz] [pdf]
[home page] [publications]