What structure does FA capture?
Factor analysis only captures pairwise
correlations between components of the data.
It only depends on the covariance matrix of
the data.
It completely ignores higher-order statistics
Consider the dataset: 111, 100, 010, 001
This has no pairwise correlations but it does have
strong third order structure.