What can perceptrons do?
They can only solve tasks if the hand-coded
features convert the original task into a linearly
separable one. How difficult is this?
The N-bit parity task :
Requires N features of the form: Are at least
m bits on?
Each feature must look at all the components
of the input.
The 2-D connectedness task
requires an exponential number of features!