mlsanityData to 14 Oct 2026

Meta-Cal: Well-controlled Post-hoc Calibration by Ranking

arXiv 2105.04290 ↗First posted 10 May 2021stat.ML cs.LGcited by 33

Not on the map. Why?

Abstract (arXiv)

In many applications, it is desirable that a classifier not only makes accurate predictions, but also outputs calibrated posterior probabilities. However, many existing classifiers, especially deep neural network classifiers, tend to be uncalibrated. Post-hoc calibration is a technique to recalibrate a model by learning a calibration map. Existing approaches mostly focus on constructing calibration maps with low calibration errors, however, this quality is inadequate for a calibrator being useful. In this paper, we introduce two constraints that are worth consideration in designing a calibration map for post-hoc calibration. Then we present Meta-Cal, which is built from a base calibrator and a ranking model. Under some mild assumptions, two high-probability bounds are given with respect to these constraints. Empirical results on CIFAR-10, CIFAR-100 and ImageNet and a range of popular network architectures show our proposed method significantly outperforms the current state of the art for post-hoc multi-class classification calibration.

Key facts. Cited by 33 papers in the map (data through 14 Oct 2026). Builds on 22 of its 43 references that are in the map. First posted on arXiv 10 May 2021.

Builds on

22 of 43 references.

References · 22

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  3. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  4. 2016Densely Connected Convolutional Networkscited by 6,720
  5. 2017On Calibration of Modern Neural Networkscited by 4,960
  6. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,926
  7. 2016Wide Residual Networkscited by 3,070
  8. 2016Deep Networks with Stochastic Depthcited by 979
  9. 2017Selective Classification for Deep Neural Networkscited by 852
  10. 2019A Simple Baseline for Bayesian Uncertainty in Deep Learningcited by 560
  11. 2018Predictive Uncertainty Estimation via Prior Networkscited by 557
  12. 2017Regularizing Neural Networks by Penalizing Confident Output Distributionscited by 464
10 more
  1. 2019Beyond temperature scaling: Obtaining well-calibrated multiclass probabilities with Dirichlet calibrationcited by 364
  2. 2019Verified Uncertainty Calibrationcited by 309
  3. 2015DeepDriving: Learning Affordance for Direct Perception in Autonomous Drivingcited by 231
  4. 2020Mix-n-Match: Ensemble and Compositional Methods for Uncertainty Calibration in Deep Learningcited by 190
  5. 2020Local Temperature Scaling for Probability Calibrationcited by 95
  6. 2016Stochastic Variational Deep Kernel Learningcited by 93
  7. 2019Non-Parametric Calibration for Classificationcited by 57
  8. 2018Dirichlet-based Gaussian Processes for Large-scale Calibrated Classificationcited by 51
  9. 2020Multi-Class Uncertainty Calibration via Mutual Information Maximization-based Binningcited by 27
  10. 2016Neyman-Pearson (NP) classification algorithms and NP receiver operating characteristics (NP-ROC)cited by 23

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 3; 2022: 6; 2023: 8; 2024: 7; 2025: 5; 2026: 4.

33 papers.

Sort:

By area:

  1. 2021The Devil is in the Margin: Margin-based Label Smoothing for Network Calibrationcited by 91
  2. 2021Classifier Calibration: A survey on how to assess and improve predicted class probabilitiescited by 73
  3. 2022On Calibrating Semantic Segmentation Models: Analyses and An Algorithmcited by 25
  4. 2023RankMixup: Ranking-Based Mixup Training for Network Calibrationcited by 19
  5. 2023ACLS: Adaptive and Conditional Label Smoothing for Network Calibrationcited by 15
  6. 2022Class Adaptive Network Calibrationcited by 13
  7. 2023Jaccard Metric Losses: Optimizing the Jaccard Index with Soft Labelscited by 12
  8. 2023Consistent and Asymptotically Unbiased Estimation of Proper Calibration Errorscited by 11
  9. 2024Calib3D: Calibrating Model Preferences for Reliable 3D Scene Understandingcited by 11
  10. 2024On Calibration of Object Detectors: Pitfalls, Evaluation and Baselinescited by 11
  11. 2022Taking a Step Back with KCal: Multi-Class Kernel-Based Calibration for Deep Neural Networkscited by 9
  12. 2022On Calibration of Graph Neural Networks for Node Classificationcited by 9
21 more
  1. 2023Beyond Classification: Definition and Density-based Estimation of Calibration in Object Detectioncited by 9
  2. 2023Scaling of Class-wise Training Losses for Post-hoc Calibrationcited by 7
  3. 2024ReliOcc: Towards Reliable Semantic Occupancy Prediction via Uncertainty Learningcited by 6
  4. 2023Dice Semimetric Losses: Optimizing the Dice Score with Soft Labelscited by 5
  5. 2025MCQA-Eval: Efficient Confidence Evaluation in NLG with Gold-Standard Correctness Labelscited by 4
  6. 2025Decision from Suboptimal Classifiers: Excess Risk Pre- and Post-Calibrationcited by 3
  7. 2021On Deep Neural Network Calibration by Regularization and its Impact on Refinementcited by 2
  8. 2022Calibrating Segmentation Networks with Margin-based Label Smoothingcited by 2
  9. 2025We Care Each Pixel: Calibrating on Medical Segmentation Modelcited by 2
  10. 2023Multiclass Alignment of Confidence and Certainty for Network Calibrationcited by 1
  11. 2024Tilt and Average : Geometric Adjustment of the Last Layer for Recalibrationcited by 1
  12. 2024Towards Precise Prediction Uncertainty in GNNs: Refining GNNs with Topology-grouping Strategycited by 1
  13. 2025Probability Bounding: Post-Hoc Calibration via Box-Constrained Softmaxcited by 1
  14. 2022Calibrating AI Models for Wireless Communications via Conformal Predictioncited by 0
  15. 2024Task-specific regularization loss towards model calibration for reliable lung cancer detectioncited by 0
  16. 2024Dual-View Pyramid Pooling in Deep Neural Networks for Improved Medical Image Classification and Confidence Calibrationcited by 0
  17. 2025Revisiting Reweighted Risk for Calibration: AURC, Focal, and Inverse Focal Losscited by 0
  18. 2026Still image and spatial-temporal tomato data enabling detection, segmentation, tracking, and video-instance segmentation using strong and weak labelscited by 0
  19. 2026ST-LoRA: Single Trajectory LoRA Ensemble for Uncertainty Aware Agricultural Segmentationcited by 0
  20. 2026Let Confidence Change, Not the Prediction: Prediction-Preserving Repair for Post-hoc Calibrationcited by 0
  21. 2026Signal-Routed Temperature Scaling: Low-Capacity Risk-Conditioned Calibration for Small Validation Budgetscited by 0