# Parallel gradient boosting for flexible estimation of conditional distributions

arXiv: https://arxiv.org/abs/2607.13550 · HTML: https://mlsanity.com/paper/2607.13550/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 296 references that are in the map.
- First posted on arXiv 15 Jul 2026.

## Builds on (most cited first, 14 of 14)

- [Conditional Generative Adversarial Nets](https://mlsanity.com/paper/1411.1784/) (2014)
- [Conditional Density Estimation with Neural Networks: Best Practices and Benchmarks](https://mlsanity.com/paper/1903.00954/) (2019)
- [XGBoostLSS -- An extension of XGBoost to probabilistic forecasting](https://mlsanity.com/paper/1907.03178/) (2019)
- [Distributional Gradient Boosting Machines](https://mlsanity.com/paper/2204.00778/) (2022)
- [MT-GBM: A Multi-Task Gradient Boosting Machine with Shared Decision Trees](https://mlsanity.com/paper/2201.06239/) (2022)
- [Component-Wise Boosting of Targets for Multi-Output Prediction](https://mlsanity.com/paper/1904.03943/) (2019)
- [Gradient tree boosting with random output projections for multi-label classification and multi-output regression](https://mlsanity.com/paper/1905.07558/) (2019)
- [When do Random Forests work?](https://mlsanity.com/paper/2504.12860/) (2025)
- [Estimation of conditional inequality curves and measures via estimating the conditional quantile function](https://mlsanity.com/paper/2412.20228/) (2024)
- [Efficient distributional regression trees learning algorithms for calibrated non-parametric probabilistic forecasts](https://mlsanity.com/paper/2502.05157/) (2025)
- [Transforming Conditional Density Estimation Into a Single Nonparametric Regression Task](https://mlsanity.com/paper/2511.18530/) (2025)
- [Generative and Nonparametric Approaches for Conditional Distribution Estimation: Methods, Perspectives, and Comparative Evaluations](https://mlsanity.com/paper/2601.22650/) (2026)
- [Robust multi-task boosting using clustering and local ensembling](https://mlsanity.com/paper/2602.14231/) (2026)
- [Conformalized Percentile Interval: Finite Sample Validity and Improved Conditional Performance](https://mlsanity.com/paper/2605.03233/) (2026)

## Abstract (arXiv)

Boosting is one of the most successful learning techniques for standard classification and regression tasks. Its extension to multi-output prediction problems has found an increasing number of applications in recent years. Among them is the prediction of entire conditional distributions rather than single functionals, which can often be framed as a multi-output regression problem, for example multiple quantile regression. Addressing such problems with classical implementations of boosting is computationally challenging, because usually one base model is trained for each target at every iteration. More efficient variants of boosting have been proposed to speed up training, but they tend to be tied to specific loss functions and classes of base learners, usually decision trees. In this work, we study a modification of the gradient boosting algorithm, which we call parallel gradient boosting, designed to circumvent all these limitations. The core idea is to use a common descent direction for all training observations. By doing so, only one base model is needed at each iteration, regardless of the number of targets, which allows for considerable performance gains. We establish sufficient conditions for the convergence of the algorithm, whose practical use is introduced via the multiple quantile regression setting. We show that in such a setting, it provides predictions of similar quality to state-of-the-art boosting libraries such as XGBoost, while being faster by several orders of magnitude. Then, we evaluate the properties of the resulting conditional distribution estimator, which is shown empirically to outperform other nonparametric and semiparametric estimators, especially in high-dimensional settings and in the presence of mixed and/or missing covariates.