# Bayesian Regression Tree Ensembles that Adapt to Smoothness and Sparsity

arXiv: https://arxiv.org/abs/1707.09461 · HTML: https://mlsanity.com/paper/1707.09461/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Jul 2017.

## Built on by (most cited first, 30 of 34)

- [Generalized Bayesian Additive Regression Trees Models: Beyond Conditional Conjugacy](https://mlsanity.com/paper/2202.09924/) (2022)
- [Shrinkage Bayesian Causal Forests for Heterogeneous Treatment Effects Estimation](https://mlsanity.com/paper/2102.06573/) (2021)
- [GP-BART: a novel Bayesian additive regression trees approach using Gaussian processes](https://mlsanity.com/paper/2204.02112/) (2022)
- [A Mixing Time Lower Bound for a Simplified Version of BART](https://mlsanity.com/paper/2210.09352/) (2022)
- [On Mixing Rates for Bayesian CART](https://mlsanity.com/paper/2306.00126/) (2023)
- [BARK: A Fully Bayesian Tree Kernel for Black-box Optimization](https://mlsanity.com/paper/2503.05574/) (2025)
- [Accounting for shared covariates in semi-parametric Bayesian additive regression trees](https://mlsanity.com/paper/2108.07636/) (2021)
- [Hierarchical Embedded Bayesian Additive Regression Trees](https://mlsanity.com/paper/2204.07207/) (2022)
- [SoftBart: Soft Bayesian Additive Regression Trees](https://mlsanity.com/paper/2210.16375/) (2022)
- [Bayesian CART models for insurance claims frequency](https://mlsanity.com/paper/2303.01923/) (2023)
- [From Reactive to Proactive Volatility Modeling with Hemisphere Neural Networks](https://mlsanity.com/paper/2311.16333/) (2023)
- [RJHMC-Tree for Exploration of the Bayesian Decision Tree Posterior](https://mlsanity.com/paper/2312.01577/) (2023)
- [Very fast Bayesian Additive Regression Trees on GPU](https://mlsanity.com/paper/2410.23244/) (2024)
- [Transition Matrix Representation of Trees with Transposed Convolutions](https://mlsanity.com/paper/2202.10677/) (2022)
- [Divide, Conquer, Combine Bayesian Decision Tree Sampling](https://mlsanity.com/paper/2403.18147/) (2024)
- [Ensembles of Probabilistic Regression Trees](https://mlsanity.com/paper/2406.14033/) (2024)
- [Efficient Decision Trees for Tensor Regressions](https://mlsanity.com/paper/2408.01926/) (2024)
- [Horseshoe Forests for High-Dimensional Causal Survival Analysis](https://mlsanity.com/paper/2507.22004/) (2025)
- [Bayesian Additive Distribution Regression](https://mlsanity.com/paper/2603.06462/) (2026)
- [Adaptive Bayesian Sum of Trees Model for Covariate Dependent Spectral Analysis](https://mlsanity.com/paper/2109.14677/) (2021)
- [Distributional Adaptive Soft Regression Trees](https://mlsanity.com/paper/2210.10389/) (2022)
- [Variational Boosted Soft Trees](https://mlsanity.com/paper/2302.10706/) (2023)
- [Generalized Bayesian Additive Regression Trees: Theory and Software](https://mlsanity.com/paper/2304.12505/) (2023)
- [ASBART:Accelerated Soft Bayes Additive Regression Trees](https://mlsanity.com/paper/2310.13975/) (2023)
- [Individualized Multi-Treatment Response Curves Estimation using RBF-net with Shared Neurons](https://mlsanity.com/paper/2401.16571/) (2024)
- [Bayesian Additive Regression Networks](https://mlsanity.com/paper/2404.04425/) (2024)
- [Functional BART with Shape Priors: A Bayesian Tree Approach to Constrained Functional Regression](https://mlsanity.com/paper/2502.16888/) (2025)
- [Improving Random Forests by Smoothing](https://mlsanity.com/paper/2505.06852/) (2025)
- [Generative Regression with IQ-BART](https://mlsanity.com/paper/2507.04168/) (2025)
- [An Infinite BART model](https://mlsanity.com/paper/2511.20087/) (2025)

## Abstract (arXiv)

Ensembles of decision trees are a useful tool for obtaining for obtaining flexible estimates of regression functions. Examples of these methods include gradient boosted decision trees, random forests, and Bayesian CART. Two potential shortcomings of tree ensembles are their lack of smoothness and vulnerability to the curse of dimensionality. We show that these issues can be overcome by instead considering sparsity inducing soft decision trees in which the decisions are treated as probabilistic. We implement this in the context of the Bayesian additive regression trees framework, and illustrate its promising performance through testing on benchmark datasets. We provide strong theoretical support for our methodology by showing that the posterior distribution concentrates at the minimax rate (up-to a logarithmic factor) for sparse functions and functions with additive structures in the high-dimensional regime where the dimensionality of the covariate space is allowed to grow near exponentially in the sample size. Our method also adapts to the unknown smoothness and sparsity levels, and can be implemented by making minimal modifications to existing BART algorithms.