A novel technique to prune variable selection ensembles

作者：Ren Liang Pin; Zhang Chun Xia; Xuan Hai Yan

来源：13th International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery, ICNC-FSKD 2017, 2017-07-29 To 2017-07-31.

DOI：10.1109/FSKD.2017.8393311

• Importance measure
• Linear regression models
• Reference vectors
• Selective ensembles
• Simulated datasets
• State-of-the-art algorithms
• Variable selection
• Variable selection problems

In ensemble learning field, it has been proven that selective ensemble learning (i.e., only fusing some instead of all ensemble members) can further improve the prediction ability of an ensemble machine. In this paper, we apply it in another framework, that is, variable selection problems in linear regression models. Under this situation, the main goal is to accurately detect the variables which have real influence on the response. As for the existing algorithms to construct a variable selection ensemble, they generally combine all the members to create an importance measure for each variable. In this paper, we propose to insert an additional pruning phase into a state-of-the-art algorithm ST2E [14]. By defining a reference vector, we sort the members generated by ST2E according to the angle between each of them and the reference vector. Then, a subensemble is obtained by only keeping some members ranked ahead. We investigated the performance of the proposed method on several simulated data sets. The experimental results show that it performs better than the original full ensemble as well as several other rivals.

• 出版日期2017
• 单位郑州大学