next up previous contents
Next: AIC and related methods Up: Bayesian methods Previous: Simulation methods   Contents


Model averaging

Aitchison (1975) [2] Shows that a suitable mixture (infinite, but in principle applies also to finite cases) of models provides a better predictive density (in terms of kullback-Leibler divergence from true model) than a signle density at a point estimate.

Smith (1986) [341] Summary of a conference talk. Emphasises the need to acknowledge that there is a range of possible models and conclusions. Comparison of posteriors across models (sensitivity); marginal (model-averaging) posterior; Bayes factors for comparison.

Moulton (1991) [279] Bayesian model selection and model averaging for econometric audience, normal linear models. Informative Cauchy priors for parameters of ineterest, improper priors for nuisance (common) parameters. Econometric example.

George and McCullogh (1993) [158] Variable selection in linear regression. Uses Gibbs sampling to create a sample of (latent) model indicators from their posterior distribution. The highest-frequency subsets are then selected for further analysis. (I.e. model averaging without the averaging step.)

Draper (1995) [113] Discussion paper on model averaging, both continuous and (mainly) discrete. Spacification of the set of models. Focus on prediction. Two major examples. Long discussion, see e.g. Box, Cox and Tukey.

Madigan and York (1995) [253] Discussion and examples of Bayesian graphical models for discrete data. Uses model averaging, with estimation using MCMC methods.

Laud and Ibrahim (1996) [233] Variable selection for linear models. Specifying a prior probabilities for the models via a prior prediction of a future observation.

Raftery et al. (1996) [303] Model averaging for survival models (including Cox regression models, for which definition of BF has to be appropriately modified). BFs (Laplace approximation), Occam's window. Shows in examples that predictive performance better than for best single model. A useful discussion.

Volinsky et al. (1997) [378] Bayesian model averaging for variable selection in Cox models. Evaluated for models in an `Occam's window' found using a leaps and bounds algorithm. Data splitting to evaluate predictive performance.

George and McCullogh (1997) [159] Bayesian variables selection for normal linear regression. Long discussion of choosing the priors (including very concentrated -- but not degenerate -- priors for `zero' coefficients) and various MCMC schemes for the computations.

Clyde (1998) [82] Bayesian model averaging. Shows how for orthonormal designs results for normal linear models give approximations for posterior model probabilities for GLMs.


next up previous contents
Next: AIC and related methods Up: Bayesian methods Previous: Simulation methods   Contents
Jouni Kuha 2003-07-16