A Bayesian view of model complexity

作者：van der Linde Angelika^*

来源：Statistica Neerlandica, 2012, 66(3): 253-271.

DOI：10.1111/j.1467-9574.2011.00518.x

• effective number of parameters
• generalized degrees of freedom
• information criteria
• model comparison
• mutual information

This article addresses the problem of formally defining the effective number of parameters in a Bayesian model which is assumed to be given by a sampling distribution and a prior distribution for the parameters. The problem occurs in the derivation of information criteria for model comparison which often trade off goodness of fit and model complexity. It also arises in (frequentist) attempts to estimate the error variance in regression models with informative priors on the regression coefficients, for example, in smoothing. It is argued that model complexity can be conceptualized as a feature of the joint distribution of the observed variables and the random parameters and might be formally described by a measure of dependence. The universal and accurate estimation of terms of model complexity is a challenging problem in practice. Several well-known criteria for model comparison are interpreted and discussed along these lines.

• 出版日期2012-8