# Aggregating distribution forecasts from deep ensembles

arXiv: https://arxiv.org/abs/2204.02291 · HTML: https://mlsanity.com/paper/2204.02291/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 90 references that are in the map.
- First posted on arXiv 5 Apr 2022.

## Builds on (most cited first, 30 of 30)

- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift](https://mlsanity.com/paper/1906.02530/) (2019)
- [Variational Dropout and the Local Reparameterization Trick](https://mlsanity.com/paper/1506.02557/) (2015)
- [Deep Ensembles: A Loss Landscape Perspective](https://mlsanity.com/paper/1912.02757/) (2019)
- [BatchEnsemble: An Alternative Approach to Efficient Ensemble and Lifelong Learning](https://mlsanity.com/paper/2002.06715/) (2020)
- [Snapshot Ensembles: Train 1, get M for free](https://mlsanity.com/paper/1704.00109/) (2017)
- [Hands-on Bayesian Neural Networks -- a Tutorial for Deep Learning Users](https://mlsanity.com/paper/2007.06823/) (2020)
- [Ensemble deep learning: A review](https://mlsanity.com/paper/2104.02395/) (2021)
- [Concrete Dropout](https://mlsanity.com/paper/1705.07832/) (2017)
- [Training independent subnetworks for robust prediction](https://mlsanity.com/paper/2010.06610/) (2020)
- [Why M Heads are Better than One: Training a Diverse Ensemble of Deep Networks](https://mlsanity.com/paper/1511.06314/) (2015)
- [Uncertainty Quantification and Deep Ensembles](https://mlsanity.com/paper/2007.08792/) (2020)
- [Learning in Implicit Generative Models](https://mlsanity.com/paper/1610.03483/) (2016)
- [Neural networks for post-processing ensemble weather forecasts](https://mlsanity.com/paper/1805.09091/) (2018)
- [Masksembles for Uncertainty Estimation](https://mlsanity.com/paper/2012.08334/) (2020)
- [Deep Ensembles Work, But Are They Necessary?](https://mlsanity.com/paper/2202.06985/) (2022)
- [Kaggle forecasting competitions: An overlooked learning opportunity](https://mlsanity.com/paper/2009.07701/) (2020)
- [Distributional Random Forests: Heterogeneity Adjustment and Multivariate Distributional Regression](https://mlsanity.com/paper/2005.14458/) (2020)
- [Forecasting: theory and practice](https://mlsanity.com/paper/2012.03854/) (2020)
- [Machine learning methods for postprocessing ensemble forecasts of wind gusts: A systematic comparison](https://mlsanity.com/paper/2106.09512/) (2021)
- [Flexible Model Aggregation for Quantile Regression](https://mlsanity.com/paper/2103.00083/) (2021)
- [Fusion of Probability Density Functions](https://mlsanity.com/paper/2202.11633/) (2022)
- [Distributional Regression Forests for Probabilistic Precipitation Forecasting in Complex Terrain](https://mlsanity.com/paper/1804.02921/) (2018)
- [Combining distribution-based neural networks to predict weather forecast probabilities](https://mlsanity.com/paper/2103.14430/) (2021)
- [Deep Distribution Regression](https://mlsanity.com/paper/1903.06023/) (2019)
- [Combining predictive distributions for statistical post-processing of ensemble forecasts](https://mlsanity.com/paper/1607.08096/) (2016)
- [A framework for probabilistic weather forecast post-processing across models and lead times using machine learning](https://mlsanity.com/paper/2005.06613/) (2020)

## Built on by (most cited first, 5 of 5)

- [Generative machine learning methods for multivariate ensemble post-processing](https://mlsanity.com/paper/2211.01345/) (2022)
- [On the Calibration of Probabilistic Classifier Sets](https://mlsanity.com/paper/2205.10082/) (2022)
- [Postprocessing of Ensemble Weather Forecasts Using Permutation-invariant Neural Networks](https://mlsanity.com/paper/2309.04452/) (2023)
- [Distributional Regression U-Nets for the Postprocessing of Precipitation Ensemble Forecasts](https://mlsanity.com/paper/2407.02125/) (2024)
- [Probabilistic intraday electricity price forecasting using generative machine learning](https://mlsanity.com/paper/2506.00044/) (2025)

## Abstract (arXiv)

The importance of accurately quantifying forecast uncertainty has motivated much recent research on probabilistic forecasting. In particular, a variety of deep learning approaches has been proposed, with forecast distributions obtained as output of neural networks. These neural network-based methods are often used in the form of an ensemble, e.g., based on multiple model runs from different random initializations or more sophisticated ensembling strategies such as dropout, resulting in a collection of forecast distributions that need to be aggregated into a final probabilistic prediction. With the aim of consolidating findings from the machine learning literature on ensemble methods and the statistical literature on forecast combination, we address the question of how to aggregate distribution forecasts based on such `deep ensembles'. Using theoretical arguments and a comprehensive analysis on twelve benchmark data sets, we systematically compare probability- and quantile-based aggregation methods for three neural network-based approaches with different forecast distribution types as output. Our results show that combining forecast distributions from deep ensembles can substantially improve the predictive performance. We propose a general quantile aggregation framework for deep ensembles that allows for corrections of systematic deficiencies and performs well in a variety of settings, often superior compared to a linear combination of the forecast densities. Finally, we investigate the effects of the ensemble size and derive recommendations of aggregating distribution forecasts from deep ensembles in practice.