# Meta-Cal: Well-controlled Post-hoc Calibration by Ranking

arXiv: https://arxiv.org/abs/2105.04290 · HTML: https://mlsanity.com/paper/2105.04290/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 43 references that are in the map.
- First posted on arXiv 10 May 2021.

## Builds on (most cited first, 22 of 22)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [Wide Residual Networks](https://mlsanity.com/paper/1605.07146/) (2016)
- [Deep Networks with Stochastic Depth](https://mlsanity.com/paper/1603.09382/) (2016)
- [Selective Classification for Deep Neural Networks](https://mlsanity.com/paper/1705.08500/) (2017)
- [A Simple Baseline for Bayesian Uncertainty in Deep Learning](https://mlsanity.com/paper/1902.02476/) (2019)
- [Predictive Uncertainty Estimation via Prior Networks](https://mlsanity.com/paper/1802.10501/) (2018)
- [Regularizing Neural Networks by Penalizing Confident Output Distributions](https://mlsanity.com/paper/1701.06548/) (2017)
- [Beyond temperature scaling: Obtaining well-calibrated multiclass probabilities with Dirichlet calibration](https://mlsanity.com/paper/1910.12656/) (2019)
- [Verified Uncertainty Calibration](https://mlsanity.com/paper/1909.10155/) (2019)
- [DeepDriving: Learning Affordance for Direct Perception in Autonomous Driving](https://mlsanity.com/paper/1505.00256/) (2015)
- [Mix-n-Match: Ensemble and Compositional Methods for Uncertainty Calibration in Deep Learning](https://mlsanity.com/paper/2003.07329/) (2020)
- [Local Temperature Scaling for Probability Calibration](https://mlsanity.com/paper/2008.05105/) (2020)
- [Stochastic Variational Deep Kernel Learning](https://mlsanity.com/paper/1611.00336/) (2016)
- [Non-Parametric Calibration for Classification](https://mlsanity.com/paper/1906.04933/) (2019)
- [Dirichlet-based Gaussian Processes for Large-scale Calibrated Classification](https://mlsanity.com/paper/1805.10915/) (2018)
- [Multi-Class Uncertainty Calibration via Mutual Information Maximization-based Binning](https://mlsanity.com/paper/2006.13092/) (2020)
- [Neyman-Pearson (NP) classification algorithms and NP receiver operating characteristics (NP-ROC)](https://mlsanity.com/paper/1608.03109/) (2016)

## Built on by (most cited first, 30 of 33)

- [The Devil is in the Margin: Margin-based Label Smoothing for Network Calibration](https://mlsanity.com/paper/2111.15430/) (2021)
- [Classifier Calibration: A survey on how to assess and improve predicted class probabilities](https://mlsanity.com/paper/2112.10327/) (2021)
- [On Calibrating Semantic Segmentation Models: Analyses and An Algorithm](https://mlsanity.com/paper/2212.12053/) (2022)
- [RankMixup: Ranking-Based Mixup Training for Network Calibration](https://mlsanity.com/paper/2308.11990/) (2023)
- [ACLS: Adaptive and Conditional Label Smoothing for Network Calibration](https://mlsanity.com/paper/2308.11911/) (2023)
- [Class Adaptive Network Calibration](https://mlsanity.com/paper/2211.15088/) (2022)
- [Jaccard Metric Losses: Optimizing the Jaccard Index with Soft Labels](https://mlsanity.com/paper/2302.05666/) (2023)
- [Consistent and Asymptotically Unbiased Estimation of Proper Calibration Errors](https://mlsanity.com/paper/2312.08589/) (2023)
- [Calib3D: Calibrating Model Preferences for Reliable 3D Scene Understanding](https://mlsanity.com/paper/2403.17010/) (2024)
- [On Calibration of Object Detectors: Pitfalls, Evaluation and Baselines](https://mlsanity.com/paper/2405.20459/) (2024)
- [Taking a Step Back with KCal: Multi-Class Kernel-Based Calibration for Deep Neural Networks](https://mlsanity.com/paper/2202.07679/) (2022)
- [On Calibration of Graph Neural Networks for Node Classification](https://mlsanity.com/paper/2206.01570/) (2022)
- [Beyond Classification: Definition and Density-based Estimation of Calibration in Object Detection](https://mlsanity.com/paper/2312.06645/) (2023)
- [Scaling of Class-wise Training Losses for Post-hoc Calibration](https://mlsanity.com/paper/2306.10989/) (2023)
- [ReliOcc: Towards Reliable Semantic Occupancy Prediction via Uncertainty Learning](https://mlsanity.com/paper/2409.18026/) (2024)
- [Dice Semimetric Losses: Optimizing the Dice Score with Soft Labels](https://mlsanity.com/paper/2303.16296/) (2023)
- [MCQA-Eval: Efficient Confidence Evaluation in NLG with Gold-Standard Correctness Labels](https://mlsanity.com/paper/2502.14268/) (2025)
- [Decision from Suboptimal Classifiers: Excess Risk Pre- and Post-Calibration](https://mlsanity.com/paper/2503.18025/) (2025)
- [On Deep Neural Network Calibration by Regularization and its Impact on Refinement](https://mlsanity.com/paper/2106.09385/) (2021)
- [Calibrating Segmentation Networks with Margin-based Label Smoothing](https://mlsanity.com/paper/2209.09641/) (2022)
- [We Care Each Pixel: Calibrating on Medical Segmentation Model](https://mlsanity.com/paper/2503.05107/) (2025)
- [Multiclass Alignment of Confidence and Certainty for Network Calibration](https://mlsanity.com/paper/2309.02636/) (2023)
- [Tilt and Average : Geometric Adjustment of the Last Layer for Recalibration](https://mlsanity.com/paper/2406.10017/) (2024)
- [Towards Precise Prediction Uncertainty in GNNs: Refining GNNs with Topology-grouping Strategy](https://mlsanity.com/paper/2412.14223/) (2024)
- [Probability Bounding: Post-Hoc Calibration via Box-Constrained Softmax](https://mlsanity.com/paper/2506.10572/) (2025)
- [Calibrating AI Models for Wireless Communications via Conformal Prediction](https://mlsanity.com/paper/2212.07775/) (2022)
- [Task-specific regularization loss towards model calibration for reliable lung cancer detection](https://mlsanity.com/paper/2401.11464/) (2024)
- [Dual-View Pyramid Pooling in Deep Neural Networks for Improved Medical Image Classification and Confidence Calibration](https://mlsanity.com/paper/2408.02906/) (2024)
- [Revisiting Reweighted Risk for Calibration: AURC, Focal, and Inverse Focal Loss](https://mlsanity.com/paper/2505.23463/) (2025)
- [Still image and spatial-temporal tomato data enabling detection, segmentation, tracking, and video-instance segmentation using strong and weak labels](https://mlsanity.com/paper/2607.14934/) (2026)

## Abstract (arXiv)

In many applications, it is desirable that a classifier not only makes accurate predictions, but also outputs calibrated posterior probabilities. However, many existing classifiers, especially deep neural network classifiers, tend to be uncalibrated. Post-hoc calibration is a technique to recalibrate a model by learning a calibration map. Existing approaches mostly focus on constructing calibration maps with low calibration errors, however, this quality is inadequate for a calibrator being useful. In this paper, we introduce two constraints that are worth consideration in designing a calibration map for post-hoc calibration. Then we present Meta-Cal, which is built from a base calibrator and a ranking model. Under some mild assumptions, two high-probability bounds are given with respect to these constraints. Empirical results on CIFAR-10, CIFAR-100 and ImageNet and a range of popular network architectures show our proposed method significantly outperforms the current state of the art for post-hoc multi-class classification calibration.