# Distributional Gradient Boosting Machines

arXiv: https://arxiv.org/abs/2204.00778 · HTML: https://mlsanity.com/paper/2204.00778/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 61 references that are in the map.
- First posted on arXiv 2 Apr 2022.

## Builds on (most cited first, 23 of 23)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [Normalizing Flows for Probabilistic Modeling and Inference](https://mlsanity.com/paper/1912.02762/) (2019)
- [DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networks](https://mlsanity.com/paper/1704.04110/) (2017)
- [Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methods](https://mlsanity.com/paper/1910.09457/) (2019)
- [Generalized Random Forests](https://mlsanity.com/paper/1610.01271/) (2016)
- [Multivariate Probabilistic Time Series Forecasting via Conditioned Normalizing Flows](https://mlsanity.com/paper/2002.06103/) (2020)
- [NGBoost: Natural Gradient Boosting for Probabilistic Prediction](https://mlsanity.com/paper/1910.03225/) (2019)
- [Deep Conditional Transformation Models](https://mlsanity.com/paper/2010.07860/) (2020)
- [Probabilistic Gradient Boosting Machines for Large-Scale Probabilistic Regression](https://mlsanity.com/paper/2106.01682/) (2021)
- [Distributional Regression Forests for Probabilistic Precipitation Forecasting in Complex Terrain](https://mlsanity.com/paper/1804.02921/) (2018)
- [Deep transformation models: Tackling complex regression problems with neural network based transformation models](https://mlsanity.com/paper/2004.00464/) (2020)
- [Gradient and Newton Boosting for Classification and Regression](https://mlsanity.com/paper/1808.03064/) (2018)
- [M5 Competition Uncertainty: Overdispersion, distributional forecasting, GAMLSS and beyond](https://mlsanity.com/paper/2107.06675/) (2021)
- [Multivariate Conditional Transformation Models](https://mlsanity.com/paper/1906.03151/) (2019)
- [Probabilistic Time Series Forecasts with Autoregressive Transformation Models](https://mlsanity.com/paper/2110.08248/) (2021)
- [Contrast Trees and Distribution Boosting](https://mlsanity.com/paper/1912.03785/) (2019)
- [Multivariate Probabilistic Regression with Natural Gradient Boosting](https://mlsanity.com/paper/2106.03823/) (2021)
- [Deep and interpretable regression models for ordinal outcomes](https://mlsanity.com/paper/2010.08376/) (2020)
- [Bernstein Flows for Flexible Posteriors in Variational Bayes](https://mlsanity.com/paper/2202.05650/) (2022)

## Built on by (most cited first, 6 of 6)

- [Mambular: A Sequential Model for Tabular Deep Learning](https://mlsanity.com/paper/2408.06291/) (2024)
- [Neural Additive Models for Location Scale and Shape: A Framework for Interpretable Neural Regression Beyond the Mean](https://mlsanity.com/paper/2301.11862/) (2023)
- [Sample-based Uncertainty Quantification with a Single Deterministic Neural Network](https://mlsanity.com/paper/2209.08418/) (2022)
- [Forecasting with Hyper-Trees](https://mlsanity.com/paper/2405.07836/) (2024)
- [Multivariate Uncertainty Quantification with Tomographic Quantile Forests](https://mlsanity.com/paper/2512.16383/) (2025)
- [Parallel gradient boosting for flexible estimation of conditional distributions](https://mlsanity.com/paper/2607.13550/) (2026)

## Abstract (arXiv)

We present a unified probabilistic gradient boosting framework for regression tasks that models and predicts the entire conditional distribution of a univariate response variable as a function of covariates. Our likelihood-based approach allows us to either model all conditional moments of a parametric distribution, or to approximate the conditional cumulative distribution function via Normalizing Flows. As underlying computational backbones, our framework is based on XGBoost and LightGBM. Modelling and predicting the entire conditional distribution greatly enhances existing tree-based gradient boosting implementations, as it allows to create probabilistic forecasts from which prediction intervals and quantiles of interest can be derived. Empirical results show that our framework achieves state-of-the-art forecast accuracy.