# Evaluation of Machine Learning Techniques for Forecast Uncertainty Quantification

arXiv: https://arxiv.org/abs/2111.14844 · HTML: https://mlsanity.com/paper/2111.14844/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 55 references that are in the map.
- First posted on arXiv 29 Nov 2021.

## Builds on (most cited first, 12 of 12)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Neural networks for post-processing ensemble weather forecasts](https://mlsanity.com/paper/1805.09091/) (2018)
- [Deep Learning for Post-Processing Ensemble Weather Forecasts](https://mlsanity.com/paper/2005.08748/) (2020)
- [Distributional Regression Forests for Probabilistic Precipitation Forecasting in Complex Terrain](https://mlsanity.com/paper/1804.02921/) (2018)
- [A comparison of combined data assimilation and machine learning methods for offline and online model error correction](https://mlsanity.com/paper/2107.11114/) (2021)
- [Analog ensemble data assimilation and a method for constructing analogs with variational autoencoders](https://mlsanity.com/paper/2006.01101/) (2020)
- [Controlled abstention neural networks for identifying skillful predictions for regression problems](https://mlsanity.com/paper/2104.08236/) (2021)
- [Machine Learning Techniques to Construct Patched Analog Ensembles for Data Assimilation](https://mlsanity.com/paper/2103.00318/) (2021)
- [Predicting Weather Uncertainty with Deep Convnets](https://mlsanity.com/paper/1911.00630/) (2019)
- [Using local dynamics to explain analog forecasting of chaotic systems](https://mlsanity.com/paper/2007.14216/) (2020)
- [Accuracy-Reliability Cost Function for Empirical Variance Estimation](https://mlsanity.com/paper/1803.04475/) (2018)
- [Statistical post-processing of wind speed forecasts using convolutional neural networks](https://mlsanity.com/paper/2007.04005/) (2020)

## Built on by (most cited first, 7 of 7)

- [Machine learning with data assimilation and uncertainty quantification for dynamical systems: a review](https://mlsanity.com/paper/2303.10462/) (2023)
- [SEEDS: Emulation of Weather Forecast Ensembles with Diffusion Models](https://mlsanity.com/paper/2306.14066/) (2023)
- [Online machine-learning forecast uncertainty estimation for sequential data assimilation](https://mlsanity.com/paper/2305.08874/) (2023)
- [Probabilistic bias adjustment of seasonal predictions of Arctic Sea Ice Concentration](https://mlsanity.com/paper/2510.09891/) (2025)
- [Toward generative machine learning for boosting ensembles of climate simulations](https://mlsanity.com/paper/2602.06287/) (2026)
- [Hybrid machine learning data assimilation for marine biogeochemistry](https://mlsanity.com/paper/2504.05218/) (2025)
- [Probabilistic bias adjustment of seasonal forecasts using generative machine learning: A case study of Arctic sea ice predictions](https://mlsanity.com/paper/2605.29172/) (2026)

## Abstract (arXiv)

Ensemble forecasting is, so far, the most successful approach to produce relevant forecasts with an estimation of their uncertainty. The main limitations of ensemble forecasting are the high computational cost and the difficulty to capture and quantify different sources of uncertainty, particularly those associated with model errors. In this work we perform toy-model and state-of-the-art model experiments to analyze to what extent artificial neural networks (ANNs) are able to model the different sources of uncertainty present in a forecast. In particular those associated with the accuracy of the initial conditions and those introduced by the model error. We also compare different training strategies: one based on a direct training using the mean and spread of an ensemble forecast as target, the other ones rely on an indirect training strategy using an analyzed state as target in which the uncertainty is implicitly learned from the data. Experiments using the Lorenz'96 model show that the ANNs are able to emulate some of the properties of ensemble forecasts like the filtering of the most unpredictable modes and a state-dependent quantification of the forecast uncertainty. Moreover, ANNs provide a reliable estimation of the forecast uncertainty in the presence of model error. Preliminary experiments conducted with a state-of-the-art forecasting system also confirm the ability of ANNs to produce a reliable quantification of the forecast uncertainty.