 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| • |
Suppose that we
pick n datapoints and assign labels of +
|
|
or – to them at
random. If our model class (e.g. a neural
|
|
|
net with a
certain number of hidden units) is powerful
|
|
|
enough to learn any
association of labels with data, its
|
|
|
too powerful!
|
|
|
| • |
Maybe we can
characterize the power of a model class
|
|
|
by asking how
many datapoints it can learn perfectly for
|
|
|
all possible
assignments of labels.
|
|
|
|
– |
This
number of datapoints is called the Vapnik-
|
|
|
Chervonenkis
dimension.
|
|
|
|
– |
Creationism
has infinite VC dimension.
|
|