27
Pro’s and con’s of replicating the weights
Advantages                      Disadvantages
There are many less
parameters.
There is an efficient
approximate learning
procedure.
After learning, inference
of hidden states is fast
and accurate.
The model is much less
powerful than a deep
network that has different
weights in each layer.
The brain clearly uses
deep networks.