Boosting as a Monte Carlo Algorithm

A new view of majority voting as a Monte Carlo stochastic algorithm is presented in this paper. The relation between the two approaches allows Adaboost's example weighting strategy to be compared with the greedy covering strategy used for a long time in Machine Learning. Even though one may expect that the greedy strategy is very much prone to overfitting, extensive experimental results do not support this guess. The greedy strategy does not clearly show overfitting, it runs in at least one order of magnitude less time, it reaches zero error on the training set in few trials, and the error on the test set is most of the time comparable, if not lower, than that exhibited by Adaboost.