Linear methods of reducing dimensionality
PCA finds the directions that have the most variance.
By representing where each datapoint is along these
axes, we minimize the squared reconstruction error.
Linear autoencoders are equivalent to PCA
Multi-Dimensional Scaling arranges the low-dimensional
points so as to minimize the discrepancy between the
pairwise distances in the original space and the pairwise
distances in the low-D space.
MDS is equivalent to PCA (if we normalize the data right)