# RJHMC-Tree for Exploration of the Bayesian Decision Tree Posterior

arXiv: https://arxiv.org/abs/2312.01577 · HTML: https://mlsanity.com/paper/2312.01577/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 28 references that are in the map.
- First posted on arXiv 4 Dec 2023.

## Builds on (most cited first, 4 of 4)

- [A Conceptual Introduction to Hamiltonian Monte Carlo](https://mlsanity.com/paper/1701.02434/) (2017)
- [Composable Effects for Flexible and Accelerated Probabilistic Programming in NumPyro](https://mlsanity.com/paper/1912.11554/) (2019)
- [Bayesian Regression Tree Ensembles that Adapt to Smoothness and Sparsity](https://mlsanity.com/paper/1707.09461/) (2017)
- [Top-down particle filtering for Bayesian decision trees](https://mlsanity.com/paper/1303.0561/) (2013)

## Built on by (most cited first, 2 of 2)

- [Learning Decision Trees as Amortized Structure Inference](https://mlsanity.com/paper/2503.06985/) (2025)
- [Divide, Conquer, Combine Bayesian Decision Tree Sampling](https://mlsanity.com/paper/2403.18147/) (2024)

## Abstract (arXiv)

Decision trees have found widespread application within the machine learning community due to their flexibility and interpretability. This paper is directed towards learning decision trees from data using a Bayesian approach, which is challenging due to the potentially enormous parameter space required to span all tree models. Several approaches have been proposed to combat this challenge, with one of the more successful being Markov chain Monte Carlo (MCMC) methods. The efficacy and efficiency of MCMC methods fundamentally rely on the quality of the so-called proposals, which is the focus of this paper. In particular, this paper investigates using a Hamiltonian Monte Carlo (HMC) approach to explore the posterior of Bayesian decision trees more efficiently by exploiting the geometry of the likelihood within a global update scheme. Two implementations of the novel algorithm are developed and compared to existing methods by testing against standard datasets in the machine learning and Bayesian decision tree literature. HMC-based methods are shown to perform favourably with respect to predictive test accuracy, acceptance rate, and tree complexity.