Over-complete ICA
using an energy-based model
Causal over-complete models preserve the
unconditional independence of the sources and
abandon the conditional independence.
Energy-based overcomplete models preserve
the conditional independence (which makes
perception fast) and abandon the unconditional
independence.
Over-complete EBM’s are easy if we use
contrastive divergence to deal with the
intractable partition function.