27
Pro’s and con’s of replicating the weights
Advantages                      Disadvantages
• There are many less
parameters.
• There is an efficient
approximate learning
procedure.
• After learning, inference
of hidden states is fast
and accurate.
• The model is much less
powerful than a deep
network that has different
weights in each layer.
• The brain clearly uses
deep networks.