mlsanityData to 7 Oct 2026

Accurate Uncertainties for Deep Learning Using Calibrated Regression

arXiv 1807.00263 ↗First posted 1 Jul 2018cs.LG stat.MLcited by 417

Not on the map. Why?

Abstract (arXiv)

Methods for reasoning under uncertainty are a key building block of accurate and reliable machine learning systems. Bayesian methods provide a general framework to quantify uncertainty. However, because of model misspecification and the use of approximate inference, Bayesian uncertainty estimates are often inaccurate -- for example, a 90% credible interval may not contain the true outcome 90% of the time. Here, we propose a simple procedure for calibrating any regression algorithm; when applied to Bayesian and probabilistic models, it is guaranteed to produce calibrated uncertainty estimates given enough data. Our procedure is inspired by Platt scaling and extends previous work on classification. We evaluate this approach on Bayesian linear regression, feedforward, and recurrent neural networks, and find that it consistently outputs well-calibrated credible intervals while improving performance on time series forecasting and model-based reinforcement learning tasks.

Key facts. Cited by 417 papers in the map (data through 7 Oct 2026). First posted on arXiv 1 Jul 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2018: 1; 2019: 0; 2020: 1; 2021: 68; 2022: 64; 2023: 84; 2024: 62; 2025: 59; 2026: 78.

417 papers.

Sort:

By area:

  1. 2023Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMscited by 946
  2. 2023Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generationcited by 795
  3. 2021A Survey of Uncertainty in Deep Neural Networkscited by 675
  4. 2020MOPO: Model-based Offline Policy Optimizationcited by 630
  5. 2018Scalable agent alignment via reward modeling: a research directioncited by 438
  6. 2021Laplace Redux -- Effortless Bayesian Deep Learningcited by 359
  7. 2021Categorical Depth Distribution Network for Monocular 3D Object Detectioncited by 326
  8. 2021Unsolved Problems in ML Safetycited by 291
  9. 2021On Calibration and Out-of-domain Generalizationcited by 134
  10. 2023Quantifying Uncertainty in Answers from any Language Model and Enhancing their Trustworthinesscited by 121
  11. 2021Truthful AI: Developing and governing AI that does not liecited by 109
  12. 2022Uncertainty Quantification with Pre-trained Language Models: A Large-Scale Empirical Analysiscited by 104
38 more
  1. 2021Calibrating Predictions to Decisions: A Novel Approach to Multi-Class Calibrationcited by 79
  2. 2023Uncertainty Quantification over Graph with Conformalized Graph Neural Networkscited by 79
  3. 2021Classifier Calibration: A survey on how to assess and improve predicted class probabilitiescited by 71
  4. 2021When and How Mixup Improves Calibrationcited by 59
  5. 2021Uncertainty Toolbox: an Open-Source Library for Assessing, Visualizing, and Improving Uncertainty Quantificationcited by 57
  6. 2022Conformal Prediction Under Feedback Covariate Shift for Biomolecular Designcited by 53
  7. 2024Conformal Prediction for Natural Language Processing: A Surveycited by 52
  8. 2021Revisiting Design Choices in Offline Model-Based Reinforcement Learningcited by 50
  9. 2021Top-label calibration and multiclass-to-binary reductionscited by 46
  10. 2023Uncertainty in Natural Language Processing: Sources, Quantification, and Applicationscited by 44
  11. 2021Survival Regression with Proper Scoring Rules and Monotonic Neural Networkscited by 42
  12. 2021PAC Prediction Sets Under Covariate Shiftcited by 42
  13. 2022The Unreasonable Effectiveness of Deep Evidential Regressioncited by 40
  14. 2022What Makes Graph Neural Networks Miscalibrated?cited by 39
  15. 2021Efficient Model-Based Multi-Agent Mean-Field Reinforcement Learningcited by 38
  16. 2021Estimating and Evaluating Regression Predictive Uncertainty in Deep Object Detectorscited by 37
  17. 2022Faithful Heteroscedastic Regression with Neural Networkscited by 36
  18. 2021Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safetycited by 35
  19. 2021Uncertainty-Aware Machine Translation Evaluationcited by 35
  20. 2021Calibrated and Sharp Uncertainties in Deep Learning via Density Estimationcited by 33
  21. 2022NeuralUQ: A comprehensive library for uncertainty quantification in neural differential equations and operatorscited by 33
  22. 2022Predictive Inference with Feature Conformal Predictioncited by 33
  23. 2023Optimistic Active Exploration of Dynamical Systemscited by 32
  24. 2022Calibrated Selective Classificationcited by 30
  25. 2021Machine learning pipeline for battery state of health estimationcited by 28
  26. 2021Valid prediction intervals for regression problemscited by 28
  27. 2022Conformal Prediction Sets with Limited False Positivescited by 28
  28. 2023Uncertainty Quantification in Machine Learning for Engineering Design and Health Prognostics: A Tutorialcited by 27
  29. 2021Contextual Dropout: An Efficient Sample-Dependent Dropout Modulecited by 26
  30. 2022Sample Efficient Deep Reinforcement Learning via Uncertainty Estimationcited by 26
  31. 2022BaCaDI: Bayesian Causal Discovery with Unknown Interventionscited by 26
  32. 2023A Large-Scale Study of Probabilistic Calibration in Neural Network Regressioncited by 26
  33. 2022Uncertainty-Aware Adaptation for Self-Supervised 3D Human Pose Estimationcited by 25
  34. 2021Simultaneously Reconciled Quantile Forecasting of Hierarchically Related Time Seriescited by 24
  35. 2021Robustness via Cross-Domain Ensemblescited by 23
  36. 2021Meta-Learning Reliable Priors in the Function Spacecited by 23
  37. 2022BayesCap: Bayesian Identity Cap for Calibrated Uncertainty in Frozen Neural Networkscited by 23
  38. 2021Flexible Model Aggregation for Quantile Regressioncited by 22

50 most cited of 417 All 417 (JSON)