mlsanityData to 14 Oct 2026

Beyond temperature scaling: Obtaining well-calibrated multiclass probabilities with Dirichlet calibration

arXiv 1910.12656 ↗First posted 28 Oct 2019cs.LG stat.MLcited by 364

Not on the map. Why?

Abstract (arXiv)

Class probabilities predicted by most multiclass classifiers are uncalibrated, often tending towards over-confidence. With neural networks, calibration can be improved by temperature scaling, a method to learn a single corrective multiplicative factor for inputs to the last softmax layer. On non-neural models the existing methods apply binary calibration in a pairwise or one-vs-rest fashion. We propose a natively multiclass calibration method applicable to classifiers from any model class, derived from Dirichlet distributions and generalising the beta calibration method from binary classification. It is easily implemented with neural nets since it is equivalent to log-transforming the uncalibrated probabilities, followed by one linear layer and softmax. Experiments demonstrate improved probabilistic predictions according to multiple measures (confidence-ECE, classwise-ECE, log-loss, Brier score) across a wide range of datasets and classifiers. Parameters of the learned Dirichlet calibration map provide insights to the biases in the uncalibrated model.

Key facts. Cited by 364 papers in the map (data through 14 Oct 2026). First posted on arXiv 28 Oct 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 42; 2022: 55; 2023: 59; 2024: 58; 2025: 74; 2026: 76.

364 papers.

Sort:

By area:

  1. 2021A Survey of Uncertainty in Deep Neural Networkscited by 680
  2. 2021Unsolved Problems in ML Safetycited by 292
  3. 2023A Survey of Confidence Estimation and Calibration in Large Language Modelscited by 256
  4. 2021Distribution-free uncertainty quantification for classification under label shiftcited by 101
  5. 2021Soft Calibration Objectives for Neural Networkscited by 92
  6. 2021Be Confident! Towards Trustworthy Graph Neural Networks via Confidence Calibrationcited by 92
  7. 2021Calibrating Predictions to Decisions: A Novel Approach to Multi-Class Calibrationcited by 79
  8. 2023Uncertainty Quantification over Graph with Conformalized Graph Neural Networkscited by 79
  9. 2023Calibration in Deep Learning: A Survey of the State-of-the-Artcited by 75
  10. 2021Classifier Calibration: A survey on how to assess and improve predicted class probabilitiescited by 73
  11. 2021Prior and Posterior Networks: A Survey on Evidential Deep Learning Methods For Uncertainty Estimationcited by 71
  12. 2022Stop Measuring Calibration When Humans Disagreecited by 71
38 more
  1. 2022Calibrated Learning to Defer with One-vs-All Classifierscited by 69
  2. 2022Better Uncertainty Calibration via Proper Scores for Classification and Beyondcited by 65
  3. 2022Forecasting Future World Events with Neural Networkscited by 63
  4. 2022Trustworthy clinical AI solutions: a unified review of uncertainty quantification in deep learning models for medical image analysiscited by 63
  5. 2023Smooth ECE: Principled Reliability Diagrams via Kernel Smoothingcited by 63
  6. 2021Learning Noise Transition Matrix from Only Noisy Labels via Total Variation Regularizationcited by 62
  7. 2022Learning to Defer to Multiple Experts: Consistent Surrogate Losses, Confidence Calibration, and Conformal Ensemblescited by 60
  8. 2023Understanding metric-related pitfalls in image analysis validationcited by 58
  9. 2022A Stitch in Time Saves Nine: A Train-Time Regularizing Loss for Improved Neural Network Calibrationcited by 49
  10. 2024Calibration and Correctness of Language Models for Codecited by 49
  11. 2022Sample-dependent Adaptive Temperature Scaling for Improved Calibrationcited by 48
  12. 2022A Consistent and Differentiable Lp Canonical Calibration Error Estimatorcited by 47
  13. 2021Top-label calibration and multiclass-to-binary reductionscited by 46
  14. 2022Low-Degree Multicalibrationcited by 45
  15. 2023Rethinking Confidence Calibration for Failure Predictioncited by 43
  16. 2024Uncertainty in Language Models: Assessment through Rank-Calibrationcited by 41
  17. 2021Combining Human Predictions with Model Probabilities via Confusion Matrices and Calibrationcited by 40
  18. 2022What Makes Graph Neural Networks Miscalibrated?cited by 40
  19. 2021Don't Just Blame Over-parametrization for Over-confidence: Theoretical Analysis of Calibration in Binary Classificationcited by 39
  20. 2021On Model Calibration for Long-Tailed Object Detection and Instance Segmentationcited by 39
  21. 2024Thermometer: Towards Universal Calibration for Large Language Modelscited by 38
  22. 2023Towards Calibrated Robust Fine-Tuning of Vision-Language Modelscited by 35
  23. 2021Meta-Cal: Well-controlled Post-hoc Calibration by Rankingcited by 33
  24. 2022Calibrated Selective Classificationcited by 33
  25. 2023Dual Focal Loss for Calibrationcited by 33
  26. 2022Beyond calibration: estimating the grouping loss of modern neural networkscited by 32
  27. 2022AdaFocal: Calibration-aware Adaptive Focal Losscited by 32
  28. 2021Spatially Varying Label Smoothing: Capturing Uncertainty from Expert Annotationscited by 31
  29. 2022Post-hoc Uncertainty Learning using a Dirichlet Meta-Modelcited by 31
  30. 2021Meta-Calibration: Learning of Model Calibration Using Differentiable Expected Calibration Errorcited by 29
  31. 2022Towards Improving Calibration in Object Detection Under Domain Shiftcited by 26
  32. 2022On Calibrating Semantic Segmentation Models: Analyses and An Algorithmcited by 25
  33. 2025Restoring Calibration for Aligned Large Language Models: A Calibration-Aware Fine-Tuning Approachcited by 25
  34. 2021Local Calibration: Metrics and Recalibrationcited by 24
  35. 2021Estimating Expected Calibration Errorscited by 24
  36. 2022T-Cal: An optimal test for the calibration of predictive modelscited by 24
  37. 2024A Survey on the Honesty of Large Language Modelscited by 23
  38. 2022Calibration tests beyond classificationcited by 20

50 most cited of 364 All 364 (JSON)