Steve's Explanation of Boosting

Boosting is a technique for training weak models to produce a stronger combined model. Usually this technique is compared to a general rule-of-thumb for data classification, such as determining what category a news article should be put in. Rules-of-thumb for a task like this would be whether the article had the word "baseball", "commodity" or "actress" in them, each of which is a good indication of different categories of articles.

Since weak hypotheses like these ones are not usually reliable indicators of what category a sentence or article might belong to, what is needed is a good method of combining these hypotheses to more accurately model the data. What boosting does is that it assumes that each of the hypotheses is assigned a weight for each category, where the weights represent how accurately the hypothesis classifies training cases into that category. The training cases are assigned weights as well, and any misclassified cases have their weights "boosted", so that the trainer will give them more importance in the next training round.

The result of this is that misclassified cases are dealt with faster, and the resulting combination of hypotheses will be the weighted combination that most accurately represents the data.

This explanation is derived from my interpretation of Robert Shapire's papers on AdaBoost and BoosTexter.