A nonlinear least squares quasi-Newton strategy for LP-SVR hyper-parameters selection

作者：Rivas Perea Pablo^*; Cota Ruiz Juan; Rosiles Jose Gerardo

来源：International Journal of Machine Learning and Cybernetics, 2014, 5(4): 579-597.

DOI：10.1007/s13042-013-0153-9

• Support vector regression
• Hyper-parameters
• Large-scale LP-SVR

This paper studies the problem of hyper-parameters selection for a linear programming-based support vector machine for regression (LP-SVR). The proposed model is a generalized method that minimizes a linear-least squares problem using a globalization strategy, inexact computation of first order information, and an existing analytical method for estimating the initial point in the hyper-parameters space. The minimization problem consists of finding the set of hyper-parameters that minimizes any generalization error function for different problems. Particularly, this research explores the case of two-class, multi-class, and regression problems. Simulation results among standard data sets suggest that the algorithm achieves statistically insignificant variability when measuring the residual error; and when compared to other methods for hyper-parameters search, the proposed method produces the lowest root mean squared error in most cases. Experimental analysis suggests that the proposed approach is better suited for large-scale applications for the particular case of an LP-SVR. Moreover, due to its mathematical formulation, the proposed method can be extended in order to estimate any number of hyper-parameters.

• 出版日期2014-8