Ways to restrict the code
Only use a few hidden units: PCA
Only allow one hidden unit to be on: Clustering
Impose a cost for each active hidden unit: ICA
Use lots of code units but squeeze the
redundancy out of the code-space.
Give each code an energy and learn an
energy-based model that makes almost all of
the possible codes highly improbable.