# Stochastic tree ensembles for regularized nonlinear regression

arXiv: https://arxiv.org/abs/2002.03375 · HTML: https://mlsanity.com/paper/2002.03375/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Feb 2020.

## Built on by (most cited first, 14 of 14)

- [Generalized Bayesian Additive Regression Trees Models: Beyond Conditional Conjugacy](https://mlsanity.com/paper/2202.09924/) (2022)
- [GP-BART: a novel Bayesian additive regression trees approach using Gaussian processes](https://mlsanity.com/paper/2204.02112/) (2022)
- [Stochastic Tree Ensembles for Estimating Heterogeneous Effects](https://mlsanity.com/paper/2209.06998/) (2022)
- [A Mixing Time Lower Bound for a Simplified Version of BART](https://mlsanity.com/paper/2210.09352/) (2022)
- [Unsupervised tree boosting for learning probability distributions](https://mlsanity.com/paper/2101.11083/) (2021)
- [Local Gaussian process extrapolation for BART models with applications to causal inference](https://mlsanity.com/paper/2204.10963/) (2022)
- [On Mixing Rates for Bayesian CART](https://mlsanity.com/paper/2306.00126/) (2023)
- [Very fast Bayesian Additive Regression Trees on GPU](https://mlsanity.com/paper/2410.23244/) (2024)
- [Growing the Efficient Frontier on Panel Trees](https://mlsanity.com/paper/2501.16730/) (2025)
- [A Bayesian Additive Regression Tree Model for Learning Conditional Average Treatment Effects in Regression Discontinuity Designs](https://mlsanity.com/paper/2503.00326/) (2025)
- [BFTS: Thompson Sampling with Bayesian Additive Regression Trees](https://mlsanity.com/paper/2602.07767/) (2026)
- [Variational Boosted Soft Trees](https://mlsanity.com/paper/2302.10706/) (2023)
- [Bayesian Causal Forests for Longitudinal Data: Assessing the Impact of Part-Time Work on Growth in High School Mathematics Achievement](https://mlsanity.com/paper/2407.11927/) (2024)
- [Forests for Differences: Robust Causal Inference Beyond Parametric DiD](https://mlsanity.com/paper/2505.09706/) (2025)

## Abstract (arXiv)

This paper develops a novel stochastic tree ensemble method for nonlinear regression, which we refer to as XBART, short for Accelerated Bayesian Additive Regression Trees. By combining regularization and stochastic search strategies from Bayesian modeling with computationally efficient techniques from recursive partitioning approaches, the new method attains state-of-the-art performance: in many settings it is both faster and more accurate than the widely-used XGBoost algorithm. Via careful simulation studies, we demonstrate that our new approach provides accurate point-wise estimates of the mean function and does so faster than popular alternatives, such as BART, XGBoost and neural networks (using Keras). We also prove a number of basic theoretical results about the new algorithm, including consistency of the single tree version of the model and stationarity of the Markov chain produced by the ensemble version. Furthermore, we demonstrate that initializing standard Bayesian additive regression trees Markov chain Monte Carlo (MCMC) at XBART-fitted trees considerably improves credible interval coverage and reduces total run-time.