mlsanityData to 7 Oct 2026

Better Uncertainty Calibration via Proper Scores for Classification and Beyond

arXiv 2203.07835 ↗First posted 15 Mar 2022cs.LG stat.MLcited by 65

Not on the map. Why?

Abstract (arXiv)

With model trustworthiness being crucial for sensitive real-world applications, practitioners are putting more and more focus on improving the uncertainty calibration of deep neural networks. Calibration errors are designed to quantify the reliability of probabilistic predictions but their estimators are usually biased and inconsistent. In this work, we introduce the framework of proper calibration errors, which relates every calibration error to a proper score and provides a respective upper bound with optimal estimation properties. This relationship can be used to reliably quantify the model calibration improvement. We theoretically and empirically demonstrate the shortcomings of commonly used estimators compared to our approach. Due to the wide applicability of proper scores, this gives a natural extension of recalibration beyond classification.

Key facts. Cited by 65 papers in the map (data through 7 Oct 2026). Builds on 31 of its 77 references that are in the map. First posted on arXiv 15 Mar 2022.

Builds on

31 of 77 references.

References · 31

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2016Densely Connected Convolutional Networkscited by 6,691
  3. 2017On Calibration of Modern Neural Networkscited by 4,860
  4. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,590
  5. 2016Wide Residual Networkscited by 3,057
  6. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,327
  7. 2019A Simple Baseline for Bayesian Uncertainty in Deep Learningcited by 555
  8. 2017Progressive Neural Architecture Searchcited by 483
  9. 2019Measuring Calibration in Deep Learningcited by 443
  10. 2021Revisiting the Calibration of Modern Neural Networkscited by 387
  11. 2019Beyond temperature scaling: Obtaining well-calibrated multiclass probabilities with Dirichlet calibrationcited by 359
  12. 2019Verified Uncertainty Calibrationcited by 307
19 more
  1. 2020Being Bayesian, Even Just a Bit, Fixes Overconfidence in ReLU Networkscited by 227
  2. 2020Mix-n-Match: Ensemble and Compositional Methods for Uncertainty Calibration in Deep Learningcited by 190
  3. 2019Evaluating model calibration in classificationcited by 165
  4. 2020Mitigating Bias in Calibration Error Estimationcited by 100
  5. 2020Calibration of Neural Networks using Splinescited by 97
  6. 2019Unconstrained Monotonic Neural Networkscited by 91
  7. 2021Be Confident! Towards Trustworthy Graph Neural Networks via Confidence Calibrationcited by 91
  8. 2019Distribution Calibration for Regressioncited by 88
  9. 2019Calibration tests in multi-class classification: A unifying frameworkcited by 79
  10. 2021A Broad Study on the Transferability of Visual Representations with Contrastive Learningcited by 78
  11. 2020Beyond Pinball Loss: Quantile Methods for Calibrated Uncertainty Quantificationcited by 77
  12. 2015Posterior calibration and exploratory analysis for natural language processing modelscited by 75
  13. 2020Intra Order-preserving Functions for Calibration of Multi-Class Neural Networkscited by 65
  14. 2020Post-hoc Uncertainty Calibration for Domain Drift Scenarioscited by 61
  15. 2019Non-Parametric Calibration for Classificationcited by 57
  16. 2021BERT & Family Eat Word Salad: Experiments with Text Understandingcited by 44
  17. 2020Being Bayesian about Categorical Probabilitycited by 35
  18. 2014On the folded normal distributioncited by 31
  19. 2021A Geometric Perspective towards Neural Calibration via Sensitivity Decompositioncited by 19

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 12; 2024: 18; 2025: 18; 2026: 15.

65 papers.

Sort:

By area:

  1. 2022Metrics reloaded: Recommendations for image analysis validationcited by 190
  2. 2024What Large Language Models Know and What People Think They Knowcited by 109
  3. 2023Understanding metric-related pitfalls in image analysis validationcited by 57
  4. 2023Dual Focal Loss for Calibrationcited by 33
  5. 2023A Primer on Bayesian Neural Networks: Review and Debatescited by 26
  6. 2025Restoring Calibration for Aligned Large Language Models: A Calibration-Aware Fine-Tuning Approachcited by 25
  7. 2024A Survey on the Honesty of Large Language Modelscited by 21
  8. 2024Gaussian Stochastic Weight Averaging for Bayesian Low-Rank Adaptation of Large Language Modelscited by 20
  9. 2022Analysis and Comparison of Classification Metricscited by 16
  10. 2023Improving Neural Additive Models with Bayesian Principlescited by 15
  11. 2025Rethinking Early Stopping: Refine, Then Calibratecited by 13
  12. 2023On (assessing) the fairness of risk score modelscited by 12
38 more
  1. 2024Evaluating Posterior Probabilities: Decision Theory, Proper Scoring Rules, and Calibrationcited by 11
  2. 2023Minimum-Risk Recalibration of Classifierscited by 10
  3. 2023Beyond Probability Partitions: Calibrating Neural Networks with Semantic Aware Groupingcited by 10
  4. 2023Consistent and Asymptotically Unbiased Estimation of Proper Calibration Errorscited by 10
  5. 2024Information-theoretic Generalization Analysis for Expected Calibration Errorcited by 9
  6. 2025Continuous Visual Autoregressive Generation via Score Maximizationcited by 9
  7. 2025Continuous Autoregressive Language Modelscited by 9
  8. 2024Language Generation with Strictly Proper Scoring Rulescited by 8
  9. 2023Towards unraveling calibration biases in medical image analysiscited by 7
  10. 2024Shaving Weights with Occam's Razor: Bayesian Sparsification for Neural Networks Using the Marginal Likelihoodcited by 7
  11. 2024Reassessing How to Compare and Improve the Calibration of Machine Learning Modelscited by 7
  12. 2025A calibration test for evaluating set-based epistemic uncertainty representationscited by 7
  13. 2023Performance Metrics for Probabilistic Ordinal Classifierscited by 5
  14. 2023Probabilistic Self-supervised Learning via Scoring Rules Minimizationcited by 4
  15. 2024Explanatory Model Monitoring to Understand the Effects of Feature Shifts on Performancecited by 4
  16. 2024SAUC: Sparsity-Aware Uncertainty Calibration for Spatiotemporal Prediction with Graph Neural Networkscited by 4
  17. 2025Where are we with calibration under dataset shift in image classification?cited by 4
  18. 2024PAC-Bayes Analysis for Recalibration in Classificationcited by 3
  19. 2024Proper losses regret at least 1/2-ordercited by 3
  20. 2024Optimizing Estimators of Squared Calibration Errors in Classificationcited by 3
  21. 2024GETS: Ensemble Temperature Scaling for Calibration in Graph Neural Networkscited by 3
  22. 2025Random Forest Calibrationcited by 3
  23. 2025Sample Margin-Aware Recalibration of Temperature Scalingcited by 3
  24. 2024Open Challenges on Fairness of Artificial Intelligence in Medical Imaging Applicationscited by 2
  25. 2025h-calibration: Rethinking Classifier Recalibration with Probabilistic Error-Bounded Objectivecited by 2
  26. 2025Improving Perturbation-based Explanations by Understanding the Role of Uncertainty Calibrationcited by 2
  27. 2023An Operational Perspective to Fairness Interventions: Where and How to Intervenecited by 1
  28. 2024Enhanced Uncertainty Estimation in Ultrasound Image Segmentation with MSU-Netcited by 1
  29. 2024Improving Uncertainty-Error Correspondence in Deep Bayesian Medical Image Segmentationcited by 1
  30. 2025Adaptive Cumulative Mass Calibration with Conformal Predictioncited by 1
  31. 2025Sample-efficient Multiclass Calibration under ℓp Errorcited by 1
  32. 2026Brenier Isotonic Regressioncited by 1
  33. 2026Calibrating Scientific Foundation Models with Inference-Time Stochastic Attentioncited by 1
  34. 2026Hierarchies of Calibration: Classification meets Regressioncited by 1
  35. 2026Decision-Aligned Evaluation of Uncertainty Quantificationcited by 1
  36. 2024Stein Variational Newton Neural Network Ensemblescited by 0
  37. 2025Uniform convergence of the smooth calibration error and its relationship with functional gradientcited by 0
  38. 2025CARE: Confidence-aware Ratio Estimation for Medical Biomarkerscited by 0

50 most cited of 65 All 65 (JSON)