An efficient gradient-based model selection algorithm for multi-output least-squares support vector regression machines

作者：Zhu, Xinqi; Gao, Zhenghong^*

来源：Pattern Recognition Letters, 2018, 111: 16-22.

DOI：10.1016/j.patrec.2018.01.023

• Support vector machines
• Multi - output regression
• Model selection
• leave-one-out cross-validation
• Gradient descent optimization

Multi-output least-squares support vector regression machines (MLS-SVR) is proposed by Xu et al. [29] to handle multi-output regression problems. However, the prohibitive cost of model selection severely hinders MLS-SVR's application. In this paper, an efficient gradient-based model selection algorithm for MLS-SVR is proposed. Firstly, a new training algorithm for MLS-SVR is developed, which allows one to obtain the solution vector for each output independently by dealing with matrices of much lower order. Based on the new training algorithm, a new leave-one-out error estimate is derived through virtual leave-one-out cross-validation. The model selection criterion is based on the new leave-one-out error estimate and its derivatives with respect to the hyper-parameters are also derived analytically. Both the model selection criterion and its partial derivatives can be obtained straightway once a training process ended. Finally, the hyper-parameters corresponding to the lowest model selection criterion is obtained through gradient decent method. The effectiveness and generalization performance of the proposed algorithm are validated through experiments on several multi-output datasets. Experiment results show that the proposed algorithm can save computational time dramatically without losing accuracy.

• 出版日期2018-8-1
• 单位西北工业大学