A surprising relationship between Boltzmann
Machines and Sigmoid Belief Nets
Directed and undirected models seem very different.
But there is a special type of multi-layer directed model
in which it is easy to infer the posterior distribution over
the hidden units because it has complementary priors.
This special type of directed model is equivalent to an
undirected model.
At first, this equivalence just seems like a neat trick
But it leads to a very effective new learning algorithm
that allows multilayer directed nets to be learned one
layer at a time.
The new learning algorithm resembles boosting with each
layer being like a weak learner.