# A review of predictive uncertainty estimation with machine learning

arXiv: https://arxiv.org/abs/2209.08307 · HTML: https://mlsanity.com/paper/2209.08307/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Builds on 63 of its 320 references that are in the map.
- First posted on arXiv 17 Sep 2022.

## Builds on (most cited first, 30 of 63)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networks](https://mlsanity.com/paper/1704.04110/) (2017)
- [Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methods](https://mlsanity.com/paper/1910.09457/) (2019)
- [A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges](https://mlsanity.com/paper/2011.06225/) (2020)
- [Deep Learning in Neural Networks: An Overview](https://mlsanity.com/paper/1404.7828/) (2014)
- [A Survey of Uncertainty in Deep Neural Networks](https://mlsanity.com/paper/2107.03342/) (2021)
- [Financial Time Series Forecasting with Deep Learning : A Systematic Literature Review: 2005-2019](https://mlsanity.com/paper/1911.13288/) (2019)
- [Recurrent Neural Networks for Time Series Forecasting: Current Status and Future Directions](https://mlsanity.com/paper/1909.00590/) (2019)
- [Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC](https://mlsanity.com/paper/1507.04544/) (2015)
- [A comparison of some conformal quantile regression methods](https://mlsanity.com/paper/1909.05433/) (2019)
- [Probabilistic Forecasting with Temporal Convolutional Neural Network](https://mlsanity.com/paper/1906.04397/) (2019)
- [Regression Conformal Prediction with Nearest Neighbours](https://mlsanity.com/paper/1401.3880/) (2014)
- [Quantifying Uncertainty in Deep Spatiotemporal Forecasting](https://mlsanity.com/paper/2105.11982/) (2021)
- [Practical heteroskedastic Gaussian process modeling for large simulation experiments](https://mlsanity.com/paper/1611.05902/) (2016)
- [Bayesian model averaging: A systematic review and conceptual classification](https://mlsanity.com/paper/1509.08864/) (2015)
- [Conformal Prediction Bands for Multivariate Functional Data](https://mlsanity.com/paper/2106.01792/) (2021)
- [Of Quantiles and Expectiles: Consistent Scoring Functions, Choquet Representations, and Forecast Rankings](https://mlsanity.com/paper/1503.08195/) (2015)
- [Isotonic Distributional Regression](https://mlsanity.com/paper/1909.03725/) (2019)
- [Semi-Structured Distributional Regression -- Extending Structured Additive Models by Arbitrary Deep Neural Networks and Data Modalities](https://mlsanity.com/paper/2002.05777/) (2020)
- [A Survey on Epistemic (Model) Uncertainty in Supervised Learning: Recent Advances and Applications](https://mlsanity.com/paper/2111.01968/) (2021)
- [CRPS Learning](https://mlsanity.com/paper/2102.00968/) (2021)
- [Most Likely Transformations](https://mlsanity.com/paper/1508.06749/) (2015)
- [Deep Compositional Spatial Models](https://mlsanity.com/paper/1906.02840/) (2019)
- [Probabilistic Gradient Boosting Machines for Large-Scale Probabilistic Regression](https://mlsanity.com/paper/2106.01682/) (2021)
- [Local Linear Forests](https://mlsanity.com/paper/1807.11408/) (2018)
- [Empirical Frequentist Coverage of Deep Learning Uncertainty Quantification Procedures](https://mlsanity.com/paper/2010.03039/) (2020)
- [Bayesian Computing with INLA: A Review](https://mlsanity.com/paper/1604.00860/) (2016)
- [Deep Distribution Regression](https://mlsanity.com/paper/1903.06023/) (2019)
- [If You Like It, GAN It. Probabilistic Multivariate Times Series Forecast With GAN](https://mlsanity.com/paper/2005.01181/) (2020)
- [Boosting algorithms in energy research: A systematic review](https://mlsanity.com/paper/2004.07049/) (2020)

## Built on by (most cited first, 29 of 29)

- [Epistemic Uncertainty in Conformal Scores: A Unified Approach](https://mlsanity.com/paper/2502.06995/) (2025)
- [Comparison of machine learning algorithms for merging gridded satellite and earth-observed precipitation data](https://mlsanity.com/paper/2301.01252/) (2022)
- [Merging satellite and gauge-measured precipitation using LightGBM with an emphasis on extreme quantiles](https://mlsanity.com/paper/2302.03606/) (2023)
- [MVG-CRPS: A Robust Loss Function for Multivariate Probabilistic Forecasting](https://mlsanity.com/paper/2410.09133/) (2024)
- [Comparison of tree-based ensemble algorithms for merging satellite and earth-observed precipitation data at the daily time scale](https://mlsanity.com/paper/2301.01214/) (2022)
- [Deep Huber quantile regression networks](https://mlsanity.com/paper/2306.10306/) (2023)
- [Ensemble learning for blending gridded satellite and gauge-measured precipitation data](https://mlsanity.com/paper/2307.06840/) (2023)
- [Ensemble learning for uncertainty estimation with application to the correction of satellite precipitation products](https://mlsanity.com/paper/2403.10567/) (2024)
- [Uncertainty estimation of machine learning spatial precipitation predictions from satellite data](https://mlsanity.com/paper/2311.07511/) (2023)
- [Bayesian Conformal Prediction as a Decision Risk Problem](https://mlsanity.com/paper/2602.03331/) (2026)
- [Understanding and mitigating difficulties in posterior predictive evaluation](https://mlsanity.com/paper/2405.19747/) (2024)
- [Superior Scoring Rules for Probabilistic Evaluation of Single-Label Multi-Class Classification Tasks](https://mlsanity.com/paper/2407.17697/) (2024)
- [Uncertainty in Supply Chain Digital Twins: A Quantum-Classical Hybrid Approach](https://mlsanity.com/paper/2411.10254/) (2024)
- [Quantile deep learning models for multi-step ahead time series prediction](https://mlsanity.com/paper/2411.15674/) (2024)
- [Multi-site modelling and reconstruction of past extreme skew surges along the French Atlantic coast](https://mlsanity.com/paper/2505.00835/) (2025)
- [Multivariate Uncertainty Quantification with Tomographic Quantile Forests](https://mlsanity.com/paper/2512.16383/) (2025)
- [Personalised dynamic super learning: an application in predicting hemodiafiltration convection volumes](https://mlsanity.com/paper/2310.08479/) (2023)
- [Combinations of distributional regression algorithms with application in uncertainty estimation of corrected satellite precipitation products](https://mlsanity.com/paper/2407.01623/) (2024)
- [Variable transformations in consistent loss functions](https://mlsanity.com/paper/2502.16542/) (2025)
- [Forests of Uncertaint(r)ees: Using tree-based ensembles to estimate probability distributions of future conflict](https://mlsanity.com/paper/2512.06210/) (2025)
- [Adaptive Conformal Prediction via Bayesian Uncertainty Weighting for Hierarchical Healthcare Data](https://mlsanity.com/paper/2601.01223/) (2026)
- [Adaptive Conformal Prediction for Reliable and Explainable Medical Image Classification](https://mlsanity.com/paper/2605.12917/) (2026)
- [Uncertainty-aware classification and triage of structural heart disease using electrocardiography and echocardiography metrics](https://mlsanity.com/paper/2605.22968/) (2026)
- [Consistent but Miscalibrated: Evaluating LLM Limitations for Risk Communication in Natural Language](https://mlsanity.com/paper/2607.03882/) (2026)
- [A Personalized Computational Framework for Assessing the Sufficiency of Partially Observed Data in Healthcare AI models](https://mlsanity.com/paper/2607.09165/) (2026)
- [Uncertainty quantification for trustworthy deep learning: Methods and measures](https://mlsanity.com/paper/2607.28248/) (2026)
- [Multimodal Deep Learning for Uncertainty-Aware Radiation Pneumonitis Risk Prediction](https://mlsanity.com/paper/2608.28937/) (2026)
- [The Role of Uncertainty in Assessing the Fairness of Machine Learning Models](https://mlsanity.com/paper/2609.07959/) (2026)
- [TimeES: Probabilistic and Deterministic Time Series Forecasting via Evolutionary Spectra](https://mlsanity.com/paper/2609.32384/) (2026)

## Abstract (arXiv)

Predictions and forecasts of machine learning models should take the form of probability distributions, aiming to increase the quantity of information communicated to end users. Although applications of probabilistic prediction and forecasting with machine learning models in academia and industry are becoming more frequent, related concepts and methods have not been formalized and structured under a holistic view of the entire field. Here, we review the topic of predictive uncertainty estimation with machine learning algorithms, as well as the related metrics (consistent scoring functions and proper scoring rules) for assessing probabilistic predictions. The review covers a time period spanning from the introduction of early statistical (linear regression and time series models, based on Bayesian statistics or quantile regression) to recent machine learning algorithms (including generalized additive models for location, scale and shape, random forests, boosting and deep learning algorithms) that are more flexible by nature. The review of the progress in the field, expedites our understanding on how to develop new algorithms tailored to users' needs, since the latest advancements are based on some fundamental concepts applied to more complex algorithms. We conclude by classifying the material and discussing challenges that are becoming a hot topic of research.