mlsanityData to 14 Oct 2026

Hierarchies of Calibration: Classification meets Regression

arXiv 2606.03245 ↗First posted 2 Jun 2026stat.ML cs.LGcited by 2

Not on the map. Why?

Abstract (arXiv)

Concepts of calibration formalize the compatibility between probabilistic predictions and the respective outcomes. In a nutshell, the outcomes ought to be indistinguishable from random draws from the predictive distributions. In this paper, we review, extend, and bridge notions of calibration that have been proposed for classification and regression tasks. Particular emphasis is given to hierarchical relations between the various notions, as they apply to general real-valued data, continuous outcomes, count data, nominal classes, and binary outcomes. To highlight a number of contributions, we introduce the notion of modal calibration for nominal outcomes, we distinguish full, partial, and average calibration in this setting, and we show that double probability integral transform (PIT) calibration is logically independent of previously proposed concepts of calibration for discrete outcomes. Furthermore, we generalize extant results on concepts of calibration that are expressed in terms of properties or functionals of the predictive distributions, such as means, quantiles, or event probabilities. Throughout the paper, we illustrate the concepts and their hierarchical relations in worked examples, and we provide algorithmic tools that support the construction of instructive examples and counterexamples.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 32 of its 76 references that are in the map. First posted on arXiv 2 Jun 2026.

Builds on

32 of 76 references.

References · 32

  1. 2022Generative Adversarial Networkscited by 5,627
  2. 2017On Calibration of Modern Neural Networkscited by 4,960
  3. 2019Measuring Calibration in Deep Learningcited by 446
  4. 2018Accurate Uncertainties for Deep Learning Using Calibrated Regressioncited by 422
  5. 2021Revisiting the Calibration of Modern Neural Networkscited by 390
  6. 2019Beyond temperature scaling: Obtaining well-calibrated multiclass probabilities with Dirichlet calibrationcited by 364
  7. 2019Verified Uncertainty Calibrationcited by 309
  8. 2019Evaluating model calibration in classificationcited by 168
  9. 2019Distribution Calibration for Regressioncited by 89
  10. 2021Calibrating Predictions to Decisions: A Novel Approach to Multi-Class Calibrationcited by 79
  11. 2021Classifier Calibration: A survey on how to assess and improve predicted class probabilitiescited by 73
  12. 2022Better Uncertainty Calibration via Proper Scores for Classification and Beyondcited by 65
20 more
  1. 2020Moment Multicalibration for Uncertainty Estimationcited by 63
  2. 2021Top-label calibration and multiclass-to-binary reductionscited by 46
  3. 2021Regression Diagnostics meets Forecast Evaluation: Conditional Calibration, Reliability Diagrams, and Coefficient of Determinationcited by 31
  4. 2019Isotonic Distributional Regressioncited by 28
  5. 2023A Large-Scale Study of Probabilistic Calibration in Neural Network Regressioncited by 27
  6. 2020Evaluating probabilistic classifiers: Reliability diagrams and score decompositions revisitedcited by 26
  7. 2023Calibration by Distribution Matching: Trainable Kernel Calibration Metricscited by 20
  8. 2018Generic Conditions for Forecast Dominancecited by 13
  9. 2022Towards Instance-Wise Calibration: Local Amortized Diagnostics and Reshaping of Conditional Densities (LADaR)cited by 10
  10. 2025In-sample calibration yields conformal calibration guaranteescited by 10
  11. 2023Decompositions of the mean continuous ranked probability scorecited by 9
  12. 2019Forecast Evaluation of Quantiles, Prediction Intervals, and other Set-Valued Functionalscited by 7
  13. 2024Tail calibration of probabilistic forecastscited by 7
  14. 2013Assessing the calibration of high-dimensional ensemble forecasts using rank histogramscited by 6
  15. 2022Score-based calibration testing for multivariate forecast distributionscited by 6
  16. 2023Assessing the calibration of multivariate probabilistic forecastscited by 5
  17. 2025Three Types of Calibration with Properties and their Semantic and Formal Relationshipscited by 5
  18. 2025Calibration through the Lens of Indistinguishabilitycited by 4
  19. 2023Double Probability Integral Transform Residuals for Regression Models with Discrete Outcomescited by 3
  20. 2025h-calibration: Rethinking Classifier Recalibration with Probabilistic Error-Bounded Objectivecited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Calibration Assessment for Nested Uncertainty Setscited by 0
  2. 2026Towards Calibrated Probabilistic Forecasts for Events of Interest via Outcome-Conditional Recalibrationcited by 0