# Tree Ensembles for Contextual Bandits

arXiv: https://arxiv.org/abs/2402.06963 · HTML: https://mlsanity.com/paper/2402.06963/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 38 references that are in the map.
- First posted on arXiv 10 Feb 2024.

## Builds on (most cited first, 14 of 14)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Revisiting Deep Learning Models for Tabular Data](https://mlsanity.com/paper/2106.11959/) (2021)
- [Deep Neural Networks and Tabular Data: A Survey](https://mlsanity.com/paper/2110.01889/) (2021)
- [Introduction to Multi-Armed Bandits](https://mlsanity.com/paper/1904.07272/) (2019)
- [A Tutorial on Thompson Sampling](https://mlsanity.com/paper/1707.02038/) (2017)
- [Learning to Optimize Via Posterior Sampling](https://mlsanity.com/paper/1301.2609/) (2013)
- [Neural Thompson Sampling](https://mlsanity.com/paper/2010.00827/) (2020)
- [Epistemic Neural Networks](https://mlsanity.com/paper/2107.08924/) (2021)
- [Ensemble Sampling](https://mlsanity.com/paper/1705.07347/) (2017)
- [Scalable Neural Contextual Bandit for Recommender Systems](https://mlsanity.com/paper/2306.14834/) (2023)
- [Random Forest for the Contextual Bandit Problem - extended version](https://mlsanity.com/paper/1504.06952/) (2015)
- [Online Learning of Energy Consumption for Navigation of Electric Vehicles](https://mlsanity.com/paper/2111.02314/) (2021)
- [Generalized Thompson Sampling for Contextual Bandits](https://mlsanity.com/paper/1310.7163/) (2013)
- [A Contextual Combinatorial Semi-Bandit Approach to Network Bottleneck Identification](https://mlsanity.com/paper/2206.08144/) (2022)

## Built on by (most cited first, 4 of 4)

- [PFN-TS: Thompson Sampling for Contextual Bandits via Prior-Data Fitted Networks](https://mlsanity.com/paper/2605.10137/) (2026)
- [Probabilistic Physics-Aware Machine Learning Predictions of Electric Truck Energy Consumption with Field Data](https://mlsanity.com/paper/2607.19054/) (2026)
- [Bandits in Prod: Hyperparameter Optimization at Inference Time](https://mlsanity.com/paper/2609.01335/) (2026)
- [Bandits via Additive Quantized Representations](https://mlsanity.com/paper/2610.02440/) (2026)

## Abstract (arXiv)

We propose a new framework for contextual multi-armed bandits based on tree ensembles. Our framework adapts two widely used bandit methods, Upper Confidence Bound and Thompson Sampling, for both standard and combinatorial settings. As part of this framework, we propose a novel method of estimating the uncertainty in tree ensemble predictions. We further demonstrate the effectiveness of our framework via several experimental studies, employing XGBoost and random forests, two popular tree ensemble methods. Compared to state-of-the-art methods based on decision trees and neural networks, our methods exhibit superior performance in terms of both regret minimization and computational runtime, when applied to benchmark datasets and the real-world application of navigation over road networks.