home page
people
research
publications
seminars
travel
search
UCL

Generative Models for Discovering Sparse Distributed Representations

Geoffrey E. Hinton and Zoubin Ghahramani
Department of Computer Science
University of Toronton

Abstract

We describe a hierarchical, generative model that can be viewed as a non-linear generalization of factor analysis and can be implemented in a neural network. The model uses bottom-up, top-down and lateral connections to perform Bayesian perceptual inference correctly. Once perceptual inference has been performed the connection strengths can be updated using a very simple learning rule that only requires locally available information. We demonstrate that the network learns to extract sparse, distributed, hierarchical representations.

Download  [ps]  [pdf]

Philosophical Transactions of the Royal Society of London, B, 352: 1177-1190

[home page]  [publications]