Ryan Prescott Adams: Learning the Structure of Deep Sparse Graphical Models

In July 2011 I moved to Harvard University to join the School of Engineering and Applied Sciences. My new web page is here.

In Proceedings of the 13th International Conference on Artificial Intelligence and Statistics (AISTATS 2010), Sardinia, Italy

Ryan Prescott Adams, Hanna M. Wallach and Zoubin Ghahramani

Deep belief networks are a powerful way to model complex probability distributions. However, it is difficult to learn the structure of a belief network, particularly one with hidden units. The Indian buffet process has been used as a nonparametric Bayesian prior on the structure of a directed belief network with a single infinitely wide hidden layer. Here, we introduce the cascading Indian buffet process (CIBP), which provides a prior on the structure of a layered, directed belief network that is unbounded in both depth and width, yet allows tractable inference. We use the CIBP prior with the nonlinear Gaussian belief network framework to allow each unit to vary its behavior between discrete and continuous representations. We use Markov chain Monte Carlo for inference in this model and explore the structures learned on image data.

This paper was awarded Best Paper at AISTATS 2010.

Errata

In the original paper, the Metropolis-Hastings acceptance ratios for the birth-death process had typos, where terms in the numerator and denominator were exchanged. These have been fixed in the above versions.