Deriving the delta rule
Define the error as the squared
residuals summed over all
training cases:
Now differentiate to get error
derivatives for weights
The batch delta rule changes
the weights in proportion to
their error derivatives summed
over all training cases