Which distributions are factorial in a
directed belief net?
• In a directed belief net with one hidden layer, the
posterior over the hidden units for each visible
vector is non-factorial (due to explaining away).
– The aggregated posterior is factorial if the
data was generated by the directed model.
• It’s the opposite way round from an undirected
model.
• The intuitions that people have from using directed
models are very misleading for undirected models.