The aims of unsupervised learning
We would like to extract a representation of the sensory
input that is useful for later processing.
We want to do this without requiring labeled data.
Prior ideas about what the internal representation should
look like ought to be helpful. So what would we like in a
representation?
Hidden causes that explain high-order correlations?
Constraints that often hold?
A low-dimensional manifold that contains all the data?
Properties that are invariant across space or time?