Boosting (ADAboost)
First train the base classifier on all the training
data with equal importance weights on each
case.
Then re-weight the training data to emphasize
the hard cases and train a second model.
How do we re-weight the data? Maybe some
theory can help.
Keep training new models on re-weighted data
Finally, use a weighted committee of all the
models for the test data.
How do we weight the models in the
committee? Maybe some theory can help.