The Matryoshka Prior and Goldilocks’ Penalization for Bayesian Regression
Andrew Womack, Indiana University
Regression has seen a resurgence in theoretical statistics due to its usefulness in providing inference for ``big data’’ problems where the number of predictors is large and potentially increasing with the number of observations. Proper control of false positives while providing good signal reconstruction relies heavily on penalizations of model complexity. In the Bayesian context, this translates to the assumption of a prior distribution on the model space. Traditional Bayesians adopt an infinitely exchangeable prior on covariate inclusion indicators, which leads to poor posterior behavior regarding false positives. Taking an entirely different approach, we choose to view each model as a local null hypothesis and the models that nest it as a set of local alternatives. We show that a simple proportionality assumption on the prior probabilities of a null and its alternatives provides a limiting Poisson distribution on model complexity, producing a sort of self tuning complexity penalization. Further, the proportionality assumption induces an isomorphism theorem on the model priors induced by conditioning on the inclusion of a set of covariates, in stark contrast to the assumptions for infinite exchangeability. This is joint work with Daniel Taylor-Rodriguez and Claudio Fuentes.