Recurrent networks
If the connectivity has directed cycles, the network can
do much more than just computing a fixed sequence of
non-linear transforms:
It can oscillate. Good for motor control?
It can settle to point attractors.
Good for classification
It can behave chaotically
But that is usually a bad idea for information processing.
It can remember things for a long time.
The network has internal state. It can decide to ignore the
input for a while if it wants to.
It can model sequential data in a natural way.
No need to use delay taps to spatialize time.