# Bayesian Additive Regression Trees using Bayesian Model Averaging

arXiv: https://arxiv.org/abs/1507.00181 · HTML: https://mlsanity.com/paper/1507.00181/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Jul 2015.

## Built on by (most cited first, 6 of 6)

- [A review of predictive uncertainty estimation with machine learning](https://mlsanity.com/paper/2209.08307/) (2022)
- [Adaptive network reliability analysis: Methodology and applications to power grid](https://mlsanity.com/paper/2109.05360/) (2021)
- [Accounting for shared covariates in semi-parametric Bayesian additive regression trees](https://mlsanity.com/paper/2108.07636/) (2021)
- [Hierarchical Embedded Bayesian Additive Regression Trees](https://mlsanity.com/paper/2204.07207/) (2022)
- [Dive into Decision Trees and Forests: A Theoretical Demonstration](https://mlsanity.com/paper/2101.08656/) (2021)
- [Co-data Learning for Bayesian Additive Regression Trees](https://mlsanity.com/paper/2311.09997/) (2023)

## Abstract (arXiv)

Bayesian Additive Regression Trees (BART) is a statistical sum of trees model. It can be considered a Bayesian version of machine learning tree ensemble methods where the individual trees are the base learners. However for data sets where the number of variables p is large (e.g. p>5,000) the algorithm can become prohibitively expensive, computationally. Another method which is popular for high dimensional data is random forests, a machine learning algorithm which grows trees using a greedy search for the best split points. However, as it is not a statistical model, it cannot produce probabilistic estimates or predictions. We propose an alternative algorithm for BART called BART-BMA, which uses Bayesian Model Averaging and a greedy search algorithm to produce a model which is much more efficient than BART for datasets with large p. BART-BMA incorporates elements of both BART and random forests to offer a model-based algorithm which can deal with high-dimensional data. We have found that BART-BMA can be run in a reasonable time on a standard laptop for the "small n large p" scenario which is common in many areas of bioinformatics. We showcase this method using simulated data and data from two real proteomic experiments; one to distinguish between patients with cardiovascular disease and controls and another to classify agressive from non-agressive prostate cancer. We compare our results to their main competitors. Open source code written in R and Rcpp to run BART-BMA can be found at: https://github.com/BelindaHernandez/BART-BMA.git