Independence relationships of hidden variables
 in three types of model
Causal            Product           Square
model             of experts           ICA
Hidden states
unconditional
on data
Hidden states
conditional on
data
independent
(generation is
easy)
dependent
(rejecting away)
independent
(by definition)
independent
(the posterior
collapses to a
single point)
independent
(inference is
easy)
dependent
(explaining away)
We now have a way to reduce this dependency
so that variational inference works