Ensemble Learning with Surrogate Splits

Abstract

Surrogate splits are used to classify test samples having missing values. In this work, they are used to produce different decisions from the same decision tree. In the popular ensemble algorithms, different sub-samples and sub-spaces are used to produce different decisions. But, in our approach, different versions of a test sample are generated by randomly deleting some features. For each version of the test sample, a different decision can be generated by using surrogate splits. 41 UCI datasets are used to compare original and surrogate split versions of the ensemble algorithms. Surrogate split versions have generally better performance than the original ones. The proposed method can be used within any ensemble algorithm using decision trees as its base learner.