mlsanityData to 7 Oct 2026

A review of predictive uncertainty estimation with machine learning

arXiv 2209.08307 ↗First posted 17 Sep 2022stat.ML cs.LG math.STcited by 29

Not on the map. Why?

Abstract (arXiv)

Predictions and forecasts of machine learning models should take the form of probability distributions, aiming to increase the quantity of information communicated to end users. Although applications of probabilistic prediction and forecasting with machine learning models in academia and industry are becoming more frequent, related concepts and methods have not been formalized and structured under a holistic view of the entire field. Here, we review the topic of predictive uncertainty estimation with machine learning algorithms, as well as the related metrics (consistent scoring functions and proper scoring rules) for assessing probabilistic predictions. The review covers a time period spanning from the introduction of early statistical (linear regression and time series models, based on Bayesian statistics or quantile regression) to recent machine learning algorithms (including generalized additive models for location, scale and shape, random forests, boosting and deep learning algorithms) that are more flexible by nature. The review of the progress in the field, expedites our understanding on how to develop new algorithms tailored to users' needs, since the latest advancements are based on some fundamental concepts applied to more complex algorithms. We conclude by classifying the material and discussing challenges that are becoming a hot topic of research.

Key facts. Cited by 29 papers in the map (data through 7 Oct 2026). Builds on 63 of its 320 references that are in the map. First posted on arXiv 17 Sep 2022.

Builds on

63 of 320 references.

References · 63

  1. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,120
  2. 2017DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networkscited by 1,004
  3. 2019Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methodscited by 903
  4. 2020A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challengescited by 813
  5. 2014Deep Learning in Neural Networks: An Overviewcited by 765
  6. 2021A Survey of Uncertainty in Deep Neural Networkscited by 675
  7. 2019Financial Time Series Forecasting with Deep Learning : A Systematic Literature Review: 2005-2019cited by 270
  8. 2019Recurrent Neural Networks for Time Series Forecasting: Current Status and Future Directionscited by 172
  9. 2015Practical Bayesian model evaluation using leave-one-out cross-validation and WAICcited by 110
  10. 2019A comparison of some conformal quantile regression methodscited by 76
  11. 2019Probabilistic Forecasting with Temporal Convolutional Neural Networkcited by 63
  12. 2014Regression Conformal Prediction with Nearest Neighbourscited by 59
38 more
  1. 2021Quantifying Uncertainty in Deep Spatiotemporal Forecastingcited by 48
  2. 2016Practical heteroskedastic Gaussian process modeling for large simulation experimentscited by 38
  3. 2015Bayesian model averaging: A systematic review and conceptual classificationcited by 29
  4. 2015Of Quantiles and Expectiles: Consistent Scoring Functions, Choquet Representations, and Forecast Rankingscited by 28
  5. 2019Isotonic Distributional Regressioncited by 28
  6. 2021Conformal Prediction Bands for Multivariate Functional Datacited by 28
  7. 2020Semi-Structured Distributional Regression -- Extending Structured Additive Models by Arbitrary Deep Neural Networks and Data Modalitiescited by 24
  8. 2021A Survey on Epistemic (Model) Uncertainty in Supervised Learning: Recent Advances and Applicationscited by 24
  9. 2021CRPS Learningcited by 21
  10. 2015Most Likely Transformationscited by 20
  11. 2019Deep Compositional Spatial Modelscited by 20
  12. 2021Probabilistic Gradient Boosting Machines for Large-Scale Probabilistic Regressioncited by 20
  13. 2018Local Linear Forestscited by 19
  14. 2020Empirical Frequentist Coverage of Deep Learning Uncertainty Quantification Procedurescited by 19
  15. 2019Deep Distribution Regressioncited by 16
  16. 2016Bayesian Computing with INLA: A Reviewcited by 14
  17. 2020If You Like It, GAN It. Probabilistic Multivariate Times Series Forecast With GANcited by 14
  18. 2020Boosting algorithms in energy research: A systematic reviewcited by 13
  19. 2021deepregression: a Flexible Neural Network Framework for Semi-Structured Deep Distributional Regressioncited by 13
  20. 2022A review of machine learning concepts and methods for addressing challenges in probabilistic hydrological post-processing and forecastingcited by 12
  21. 2021M5 Competition Uncertainty: Overdispersion, distributional forecasting, GAMLSS and beyondcited by 11
  22. 2015Comparison of Bayesian predictive methods for model selectioncited by 10
  23. 2019Building Calibrated Deep Models via Uncertainty Matching with Auxiliary Interval Predictorscited by 10
  24. 2016Stability selection for component-wise gradient boosting in multiple dimensionscited by 9
  25. 2018Prediction based on conditional distributions of vine copulascited by 8
  26. 2021Probabilistic Time Series Forecasts with Autoregressive Transformation Modelscited by 7
  27. 2022Deselection of Base-Learners for Statistical Boosting -- with an Application to Distributional Regressioncited by 7
  28. 2015Bayesian Nonparametric Calibration and Combination of Predictive Distributionscited by 6
  29. 2015Bayesian Additive Regression Trees using Bayesian Model Averagingcited by 6
  30. 2016Extremal Quantile Regression: An Overviewcited by 6
  31. 2019Marginally-calibrated deep distributional regressioncited by 6
  32. 2019Focused Bayesian Predictioncited by 6
  33. 2020Distributional (Single) Index Modelscited by 6
  34. 2015D-vine copula based quantile regressioncited by 5
  35. 2018Practical Bayesian Modeling and Inference for Massive Spatial Datasets On Modest Computing Environmentscited by 5
  36. 2019Contrast Trees and Distribution Boostingcited by 5
  37. 2020Quantile Regression Neural Networks: A Bayesian Approachcited by 5
  38. 2021Nonparametric C- and D-vine based quantile regressioncited by 5

50 most cited of 63 All 63 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 5; 2024: 7; 2025: 5; 2026: 10.

29 papers.

Sort:

By area:

  1. 2025Epistemic Uncertainty in Conformal Scores: A Unified Approachcited by 9
  2. 2022Comparison of machine learning algorithms for merging gridded satellite and earth-observed precipitation datacited by 6
  3. 2023Merging satellite and gauge-measured precipitation using LightGBM with an emphasis on extreme quantilescited by 5
  4. 2024MVG-CRPS: A Robust Loss Function for Multivariate Probabilistic Forecastingcited by 5
  5. 2022Comparison of tree-based ensemble algorithms for merging satellite and earth-observed precipitation data at the daily time scalecited by 4
  6. 2023Deep Huber quantile regression networkscited by 3
  7. 2023Ensemble learning for blending gridded satellite and gauge-measured precipitation datacited by 3
  8. 2024Ensemble learning for uncertainty estimation with application to the correction of satellite precipitation productscited by 3
  9. 2023Uncertainty estimation of machine learning spatial precipitation predictions from satellite datacited by 2
  10. 2026Bayesian Conformal Prediction as a Decision Risk Problemcited by 2
  11. 2024Understanding and mitigating difficulties in posterior predictive evaluationcited by 1
  12. 2024Superior Scoring Rules for Probabilistic Evaluation of Single-Label Multi-Class Classification Taskscited by 1
17 more
  1. 2024Uncertainty in Supply Chain Digital Twins: A Quantum-Classical Hybrid Approachcited by 1
  2. 2024Quantile deep learning models for multi-step ahead time series predictioncited by 1
  3. 2025Multi-site modelling and reconstruction of past extreme skew surges along the French Atlantic coastcited by 1
  4. 2025Multivariate Uncertainty Quantification with Tomographic Quantile Forestscited by 1
  5. 2023Personalised dynamic super learning: an application in predicting hemodiafiltration convection volumescited by 0
  6. 2024Combinations of distributional regression algorithms with application in uncertainty estimation of corrected satellite precipitation productscited by 0
  7. 2025Variable transformations in consistent loss functionscited by 0
  8. 2025Forests of Uncertaint(r)ees: Using tree-based ensembles to estimate probability distributions of future conflictcited by 0
  9. 2026Adaptive Conformal Prediction via Bayesian Uncertainty Weighting for Hierarchical Healthcare Datacited by 0
  10. 2026Adaptive Conformal Prediction for Reliable and Explainable Medical Image Classificationcited by 0
  11. 2026Uncertainty-aware classification and triage of structural heart disease using electrocardiography and echocardiography metricscited by 0
  12. 2026Consistent but Miscalibrated: Evaluating LLM Limitations for Risk Communication in Natural Languagecited by 0
  13. 2026A Personalized Computational Framework for Assessing the Sufficiency of Partially Observed Data in Healthcare AI modelscited by 0
  14. 2026Uncertainty quantification for trustworthy deep learning: Methods and measurescited by 0
  15. 2026Multimodal Deep Learning for Uncertainty-Aware Radiation Pneumonitis Risk Predictioncited by 0
  16. 2026The Role of Uncertainty in Assessing the Fairness of Machine Learning Modelscited by 0
  17. 2026TimeES: Probabilistic and Deterministic Time Series Forecasting via Evolutionary Spectracited by 0