Local Boosting of Decision Stumps for Regression and Classification Problems

Abstract

Numerous data mining problems involve an investigation of associations between features in heterogeneous datasets, where different prediction models can be more suitable for different regions. We propose a technique of boosting localized weak learners; rather than having constant weights attached to each learner (as in standard boosting approaches), we allow weights to be functions over the input domain. In order to find out these functions, we recognize local regions having similar characteristics and then build local experts on each of these regions describing the association between the data characteristics and the target value. We performed a comparison with other well known combining methods on standard classification and regression benchmark datasets using decision stump as based learner, and the proposed technique produced the most accurate results.