mlsanityData to 14 Oct 2026

Evaluating and Calibrating Uncertainty Prediction in Regression Tasks

arXiv 1905.11659 ↗First posted 28 May 2019cs.LG stat.MLcited by 87

Not on the map. Why?

Abstract (arXiv)

Predicting not only the target but also an accurate measure of uncertainty is important for many machine learning applications and in particular safety-critical ones. In this work we study the calibration of uncertainty prediction for regression tasks which often arise in real-world systems. We show that the existing definition for calibration of a regression uncertainty [Kuleshov et al. 2018] has severe limitations in distinguishing informative from non-informative uncertainty predictions. We propose a new definition that escapes this caveat and an evaluation method using a simple histogram-based approach. Our method clusters examples with similar uncertainty prediction and compares the prediction with the empirical uncertainty on these examples. We also propose a simple, scaling-based calibration method that preforms as well as much more complex ones. We show results on both a synthetic, controlled problem and on the object detection bounding-box regression task using the COCO and KITTI datasets.

Key facts. Cited by 87 papers in the map (data through 14 Oct 2026). First posted on arXiv 28 May 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 12; 2022: 5; 2023: 27; 2024: 12; 2025: 11; 2026: 20.

87 papers.

Sort:

By area:

  1. 2021A Survey of Uncertainty in Deep Neural Networkscited by 680
  2. 2021How to Certify Machine Learning Based Safety-critical Systems? A Systematic Literature Reviewcited by 39
  3. 2022NeuralUQ: A comprehensive library for uncertainty quantification in neural differential equations and operatorscited by 33
  4. 2021Deep Ensembles from a Bayesian Perspectivecited by 27
  5. 2022Sample Efficient Deep Reinforcement Learning via Uncertainty Estimationcited by 27
  6. 2022BayesCap: Bayesian Identity Cap for Calibrated Uncertainty in Frozen Neural Networkscited by 23
  7. 2021Calibrated Uncertainty for Molecular Property Prediction using Ensembles of Message Passing Neural Networkscited by 21
  8. 2023Towards Building Self-Aware Object Detectors via Reliable Uncertainty Quantification and Calibrationcited by 19
  9. 2021Recalibration of Aleatoric and Epistemic Regression Uncertainty in Medical Imagingcited by 18
  10. 2023URL: A Representation Learning Benchmark for Transferable Uncertainty Estimatescited by 17
  11. 2023How Reliable is Your Regression Model's Uncertainty Under Real-World Distribution Shifts?cited by 14
  12. 2023Calibration in Machine Learning Uncertainty Quantification: beyond consistency to target adaptivitycited by 13
38 more
  1. 2026BAGEN: Are LLM Agents Budget-Aware?cited by 13
  2. 2024Kermut: Composite kernel regression for protein variant effectscited by 12
  3. 2021Robust Monocular Localization in Sparse HD Maps Leveraging Multi-Task Uncertainty Estimationcited by 11
  4. 2023Evidential Deep Learning: Enhancing Predictive Uncertainty Estimation for Earth System Science Applicationscited by 11
  5. 2024Diffusion MRI with Machine Learningcited by 11
  6. 2023Graph Neural Network Interatomic Potential Ensembles with Calibrated Aleatoric and Epistemic Uncertainty on Energy and Forcescited by 10
  7. 2023Understanding Pathologies of Deep Heteroskedastic Regressioncited by 10
  8. 2022Parametric and Multivariate Uncertainty Calibration for Regression and Object Detectioncited by 9
  9. 2023A Holistic Assessment of the Reliability of Machine Learning Systemscited by 9
  10. 2023Clarifying Trust of Materials Property Predictions using Neural Networks with Distribution-Specific Uncertainty Quantificationcited by 8
  11. 2023Properties of the ENCE and other MAD-based calibration metricscited by 7
  12. 2023Uncertainty Quantification in Machine Learning for Biosignal Applications -- A Reviewcited by 7
  13. 2021Reliable Neural Networks for Regression Uncertainty Estimationcited by 6
  14. 2023Stratification of uncertainties recalibrated by isotonic regression and its impact on calibration error statisticscited by 6
  15. 2023Validation of uncertainty quantification metrics: a primer based on the consistency and adaptivity conceptscited by 5
  16. 2023Conformalizing Machine Translation Evaluationcited by 5
  17. 2023Tailoring Mixup to Data for Calibrationcited by 5
  18. 2024A Survey on Lexical Ambiguity Detection and Word Sense Disambiguationcited by 5
  19. 2024Enhancing Uncertainty Quantification in Drug Discovery with Censored Regression Labelscited by 5
  20. 2024SAUC: Sparsity-Aware Uncertainty Calibration for Spatiotemporal Prediction with Graph Neural Networkscited by 5
  21. 2021A framework for benchmarking uncertainty in deep regressioncited by 4
  22. 2023Can bin-wise scaling improve consistency and adaptivity of prediction uncertainty for machine learning regression ?cited by 4
  23. 2024Multi-fidelity Gaussian process surrogate modeling for regression problems in physicscited by 4
  24. 2024Navigation under uncertainty: Trajectory prediction and occlusion reasoning with switching dynamical systemscited by 4
  25. 2021f-Cal: Calibrated aleatoric uncertainty estimation from neural networks for robot perceptioncited by 3
  26. 2023Likelihood Annealing: Fast Calibrated Uncertainty for Regressioncited by 3
  27. 2023Towards Reliable Uncertainty Quantification via Deep Ensembles in Multi-output Regression Taskcited by 3
  28. 2024denoiSplit: a method for joint microscopy image splitting and unsupervised denoisingcited by 3
  29. 2024Improved Uncertainty Estimation of Graph Neural Network Potentials Using Engineered Latent Space Distancescited by 3
  30. 2025Uncertainty quantification with approximate variational learning for wearable photoplethysmography prediction taskscited by 3
  31. 2021Evaluating Predictive Uncertainty and Robustness to Distributional Shift Using Real World Datacited by 2
  32. 2022A Robust Learning Methodology for Uncertainty-aware Scientific Machine Learning modelscited by 2
  33. 2023Surrogate uncertainty estimation for your time series forecasting black-box: learn when to trustcited by 2
  34. 2023A General Framework for Uncertainty Quantification via Neural SDE-RNNcited by 2
  35. 2023Machine-Learning Solutions for the Analysis of Single-Particle Diffusion Trajectoriescited by 2
  36. 2023HRTF Interpolation using a Spherical Neural Process Meta-Learnercited by 2
  37. 2025CLUE: Neural Networks Calibration via Learning Uncertainty-Error alignmentcited by 2
  38. 2025LFaB: Low fidelity as Bias for Active Learning in the chemical configuration spacecited by 2

50 most cited of 87 All 87 (JSON)