Multi-Objective Model Selection via Racing

作者：Zhang Tiantian^*; Georgiopoulos Michael; Anagnostopoulos Georgios C

来源：IEEE Transactions on Cybernetics, 2016, 46(8): 1863-1876.

DOI：10.1109/TCYB.2015.2456187

• Discrete Holm's step-down procedure
• model selection
• multi-objective optimization
• racing algorithm (RA)
• sign test (ST)

Model selection is a core aspect in machine learning and is, occasionally, multi-objective in nature. For instance, hyper-parameter selection in a multi-task learning context is of multi-objective nature, since all the tasks' objectives must be optimized simultaneously. In this paper, a novel multi-objective racing algorithm (RA), namely S-Race, is put forward. Given an ensemble of models, our task is to reliably identify Pareto optimal models evaluated against multiple objectives, while minimizing the total computational cost. As a RA, S-Race attempts to eliminate non-promising models with confidence as early as possible, so as to concentrate computational resources on promising ones. Simultaneously, it addresses the problem of multi-objective model selection (MOMS) in the sense of Pareto optimality. In S-Race, the nonparametric sign test is utilized for pair-wise dominance relationship identification. Moreover, a discrete Holm's step-down procedure is adopted to control the family-wise error rate of the set of hypotheses made simultaneously. The significance level assigned to each family is adjusted adaptively during the race. In order to illustrate its merits, S-Race is applied on three MOMS problems: 1) selecting support vector machines for classification; 2) tuning the parameters of artificial bee colony algorithms for numerical optimization; and 3) constructing optimal hybrid recommendation systems for movie recommendation. The experimental results confirm that S-Race is an efficient and effective MOMS algorithm compared to a brute-force approach.1,2

• 出版日期2016-8