Title
Efficient Metropolis-Hastings Proposal Mechanisms for Bayesian Regression Tree Models
Speaker
Mattew Pratola, Simon Fraser University
Abstract
Bayesian regression trees are flexible non-parametric models that are well suited to many modern statistical regression problems. Their non-parametric formulation allows for effective and efficient modeling of datasets exhibiting complex non-linear relationships between model predictors and the response. However, the mixing behavior of the MCMC sampler is sometimes poor. This poor mixing can lead to inferential problems, such as under-representing uncertainty.
In this talk, we develop two novel proposal mechanisms for efficiently sampling the regression tree posterior. The first is a rule perturbation proposal with an optional pre-conditioned change-of-variable step, while the second we call tree rotation. The perturbation proposal can be seen as an efficient variation of the usual change proposal found in existing literature. The novel tree rotation proposal is simple to implement as it only requires local changes to the regression tree structure, yet it efficiently traverses disparate regions of the tree model space. When combined with the usual birth/death proposal, the resulting MCMC sampler exhibits good acceptance rates and properly represents model uncertainty in the posterior samples. We implement this sampling algorithm in the Bayesian Additive Regression Tree (BART) model and demonstrate its effectiveness on a prediction problem from computer experiments.
