# Multivariate Probabilistic Regression with Natural Gradient Boosting

arXiv: https://arxiv.org/abs/2106.03823 · HTML: https://mlsanity.com/paper/2106.03823/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 25 references that are in the map.
- First posted on arXiv 7 Jun 2021.

## Builds on (most cited first, 5 of 5)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Neural networks for post-processing ensemble weather forecasts](https://mlsanity.com/paper/1805.09091/) (2018)
- [NGBoost: Natural Gradient Boosting for Probabilistic Prediction](https://mlsanity.com/paper/1910.03225/) (2019)
- [Natural Gradients in Practice: Non-Conjugate Variational Inference in Gaussian Process Models](https://mlsanity.com/paper/1803.09151/) (2018)
- [Deep Distribution Regression](https://mlsanity.com/paper/1903.06023/) (2019)

## Built on by (most cited first, 5 of 5)

- [Distributional Gradient Boosting Machines](https://mlsanity.com/paper/2204.00778/) (2022)
- [Multi-Target XGBoostLSS Regression](https://mlsanity.com/paper/2210.06831/) (2022)
- [TreeFlow: Going beyond Tree-based Gaussian Probabilistic Regression](https://mlsanity.com/paper/2206.04140/) (2022)
- [Multivariate Uncertainty Quantification with Tomographic Quantile Forests](https://mlsanity.com/paper/2512.16383/) (2025)
- [Anchor-MoE: A Mean-Anchored Mixture of Experts For Probabilistic Regression](https://mlsanity.com/paper/2508.16802/) (2025)

## Abstract (arXiv)

Many single-target regression problems require estimates of uncertainty along with the point predictions. Probabilistic regression algorithms are well-suited for these tasks. However, the options are much more limited when the prediction target is multivariate and a joint measure of uncertainty is required. For example, in predicting a 2D velocity vector a joint uncertainty would quantify the probability of any vector in the plane, which would be more expressive than two separate uncertainties on the x- and y- components. To enable joint probabilistic regression, we propose a Natural Gradient Boosting (NGBoost) approach based on nonparametrically modeling the conditional parameters of the multivariate predictive distribution. Our method is robust, works out-of-the-box without extensive tuning, is modular with respect to the assumed target distribution, and performs competitively in comparison to existing approaches. We demonstrate these claims in simulation and with a case study predicting two-dimensional oceanographic velocity data. An implementation of our method is available at https://github.com/stanfordmlgroup/ngboost.