# TreeFlow: Going beyond Tree-based Gaussian Probabilistic Regression

arXiv: https://arxiv.org/abs/2206.04140 · HTML: https://mlsanity.com/paper/2206.04140/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 28 references that are in the map.
- First posted on arXiv 8 Jun 2022.

## Builds on (most cited first, 21 of 21)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [Density estimation using Real NVP](https://mlsanity.com/paper/1605.08803/) (2016)
- [Glow: Generative Flow with Invertible 1x1 Convolutions](https://mlsanity.com/paper/1807.03039/) (2018)
- [FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Models](https://mlsanity.com/paper/1810.01367/) (2018)
- [PointFlow: 3D Point Cloud Generation with Continuous Normalizing Flows](https://mlsanity.com/paper/1906.12320/) (2019)
- [NGBoost: Natural Gradient Boosting for Probabilistic Prediction](https://mlsanity.com/paper/1910.03225/) (2019)
- [Uncertainty in Gradient Boosting via Ensembles](https://mlsanity.com/paper/2006.10562/) (2020)
- [Transforming Gaussian Processes With Normalizing Flows](https://mlsanity.com/paper/2011.01596/) (2020)
- [Conditional Sum-Product Networks: Imposing Structure on Deep Probabilistic Architectures](https://mlsanity.com/paper/1905.08550/) (2019)
- [Probabilistic Gradient Boosting Machines for Large-Scale Probabilistic Regression](https://mlsanity.com/paper/2106.01682/) (2021)
- [TraDE: Transformers for Density Estimation](https://mlsanity.com/paper/2004.02441/) (2020)
- [XGBoostLSS -- An extension of XGBoost to probabilistic forecasting](https://mlsanity.com/paper/1907.03178/) (2019)
- [Non-Gaussian Gaussian Processes for Few-Shot Regression](https://mlsanity.com/paper/2110.13561/) (2021)
- [Leveraging Probabilistic Circuits for Nonparametric Multi-Output Regression](https://mlsanity.com/paper/2106.08687/) (2021)
- [RegFlow: Probabilistic Flow-based Regression for Future Prediction](https://mlsanity.com/paper/2011.14620/) (2020)
- [Multivariate Probabilistic Regression with Natural Gradient Boosting](https://mlsanity.com/paper/2106.03823/) (2021)
- [CatBoostLSS -- An extension of CatBoost to probabilistic forecasting](https://mlsanity.com/paper/2001.02121/) (2020)
- [RoNGBa: A Robustly Optimized Natural Gradient Boosting Training Approach with Leaf Number Clipping](https://mlsanity.com/paper/1912.02338/) (2019)

## Built on by (most cited first, 1 of 1)

- [TabResFlow: A Normalizing Spline Flow Model for Probabilistic Univariate Tabular Regression](https://mlsanity.com/paper/2508.17056/) (2025)

## Abstract (arXiv)

The tree-based ensembles are known for their outstanding performance in classification and regression problems characterized by feature vectors represented by mixed-type variables from various ranges and domains. However, considering regression problems, they are primarily designed to provide deterministic responses or model the uncertainty of the output with Gaussian or parametric distribution. In this work, we introduce TreeFlow, the tree-based approach that combines the benefits of using tree ensembles with the capabilities of modeling flexible probability distributions using normalizing flows. The main idea of the solution is to use a tree-based model as a feature extractor and combine it with a conditional variant of normalizing flow. Consequently, our approach is capable of modeling complex distributions for the regression outputs. We evaluate the proposed method on challenging regression benchmarks with varying volume, feature characteristics, and target dimensionality. We obtain the SOTA results for both probabilistic and deterministic metrics on datasets with multi-modal target distributions and competitive results on unimodal ones compared to tree-based regression baselines.