The deep autoencoder
784
400
200
100
50
25
6
linear units
784
400
200
100
50
25
If you start with small random weights it will not
learn.
If you break symmetry randomly by using
bigger weights, it will not find a good solution.