| The information maximization view of ICA |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| • | Filter the data
linearly and then applying a non- |
|||||||||||||||||
| linear
“squashing” function. |
||||||||||||||||||
| • | The aim is to
maximize the information that the |
|||||||||||||||||
| outputs convey
about the input. |
||||||||||||||||||
| – | Since
the outputs are a deterministic function |
|||||||||||||||||
| of
the inputs, information is maximized by |
||||||||||||||||||
| maximizing
the entropy of the output |
||||||||||||||||||
| distribution. |
||||||||||||||||||
| • | This
involves maximizing the individual entropies |
|||||||||||||||||
| of the
outputs and minimizing the mutual |
||||||||||||||||||
| information
between outputs. |
||||||||||||||||||