A Generic Path Algorithm for Regularized Statistical Estimation

作者：Zhou Hua^*; Wu Yichao

来源：Journal of the American Statistical Association, 2014, 109(506): 686-699.

DOI：10.1080/01621459.2013.864166

• Gaussian graphical model
• Generalized linear model
• Lasso
• Log-concave density estimation
• Ordinary differential equations
• Quasi-likelihoods
• Regularization
• Shape restricted regression
• Solution path

Regularization is widely used in statistics and machine learning to prevent overfitting and gear solution toward prior information. In general, a regularized estimation problem minimizes the sum of a loss function and a penalty term. The penalty term is usually weighted by a tuning parameter and encourages certain constraints on the parameters to be estimated. Particular choices of constraints lead to the popular lasso, fused-lasso, and other generalized l(1) penalized regression methods. In this article we follow a recent idea by Wu and propose an exact path solver based on ordinary differential equations (EPSODE) that works for any convex loss function and can deal with generalized l(1) penalties as well as more complicated regularization such as inequality constraints encountered in shape-restricted regressions and nonparametric density estimation. Nonasymptotic error bounds for the equality regularized estimates are derived. In practice, the EPSODE can be coupled with AIC, BIC, C-p or cross-validation to select an optimal tuning parameter, or provide a convenient model space for performing model averaging or aggregation. Our applications to generalized l(1) regularized generalized linear models, shape-restricted regressions, Gaussian graphical models, and nonparametric density estimation showcase the potential of the EPSODE algorithm. Supplementary materials for this article are available online.

• 出版日期2014-6