# Uncertainty-Aware Extrapolation in Bayesian Oblique Trees

arXiv: https://arxiv.org/abs/2601.22899 · HTML: https://mlsanity.com/paper/2601.22899/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 4 of its 18 references that are in the map.
- First posted on arXiv 30 Jan 2026.

## Builds on (most cited first, 4 of 4)

- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [Fast Linear Model Trees by PILOT](https://mlsanity.com/paper/2302.03931/) (2023)
- [Oblique Predictive Clustering Trees](https://mlsanity.com/paper/2007.13617/) (2020)
- [A conservation law for posterior predictive variance](https://mlsanity.com/paper/2406.11806/) (2024)

## Abstract (arXiv)

Decision trees are widely used due to their interpretability and efficiency, but they struggle in regression tasks that require reliable extrapolation and well-calibrated uncertainty. Piecewise-constant leaf predictions are bounded by the training targets and often become overconfident under distribution shift. We propose a single-tree Bayesian model that extends VSPYCT by equipping each leaf with a GP predictor. Bayesian oblique splits provide uncertainty-aware partitioning of the input space, while GP leaves model local functional behaviour and enable principled extrapolation beyond the observed target range. We present an efficient inference and prediction scheme that combines posterior sampling of split parameters with glsgp posterior predictions, and a gating mechanism that activates GP-based extrapolation when inputs fall outside the training support of a leaf. Experiments on benchmark regression tasks show improvements in the predictive performance compared to standard variational oblique trees, and substantial performance gains in extrapolation scenarios.