# Intra Order-preserving Functions for Calibration of Multi-Class Neural Networks

arXiv: https://arxiv.org/abs/2003.06820 · HTML: https://mlsanity.com/paper/2003.06820/

## Key facts

- Cited by 65 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Mar 2020.

## Built on by (most cited first, 30 of 65)

- [On Calibration and Out-of-domain Generalization](https://mlsanity.com/paper/2102.10395/) (2021)
- [Be Confident! Towards Trustworthy Graph Neural Networks via Confidence Calibration](https://mlsanity.com/paper/2109.14285/) (2021)
- [Better Uncertainty Calibration via Proper Scores for Classification and Beyond](https://mlsanity.com/paper/2203.07835/) (2022)
- [Learning Noise Transition Matrix from Only Noisy Labels via Total Variation Regularization](https://mlsanity.com/paper/2102.02414/) (2021)
- [Parameterized Temperature Scaling for Boosting the Expressive Power in Post-Hoc Uncertainty Calibration](https://mlsanity.com/paper/2102.12182/) (2021)
- [Top-label calibration and multiclass-to-binary reductions](https://mlsanity.com/paper/2107.08353/) (2021)
- [Rethinking Confidence Calibration for Failure Prediction](https://mlsanity.com/paper/2303.02970/) (2023)
- [What Makes Graph Neural Networks Miscalibrated?](https://mlsanity.com/paper/2210.06391/) (2022)
- [Don't Just Blame Over-parametrization for Over-confidence: Theoretical Analysis of Calibration in Binary Classification](https://mlsanity.com/paper/2102.07856/) (2021)
- [Adaptive Temperature Scaling for Robust Calibration of Deep Neural Networks](https://mlsanity.com/paper/2208.00461/) (2022)
- [Obtaining Calibrated Probabilities with Personalized Ranking Models](https://mlsanity.com/paper/2112.07428/) (2021)
- [Beyond In-Domain Scenarios: Robust Density-Aware Calibration](https://mlsanity.com/paper/2302.05118/) (2023)
- [Natural Posterior Network: Deep Bayesian Uncertainty for Exponential Family Distributions](https://mlsanity.com/paper/2105.04471/) (2021)
- [Uncertainty Estimation for Molecules: Desiderata and Methods](https://mlsanity.com/paper/2306.14916/) (2023)
- [Consistent and Asymptotically Unbiased Estimation of Proper Calibration Errors](https://mlsanity.com/paper/2312.08589/) (2023)
- [Weighted Aggregation of Conformity Scores for Classification](https://mlsanity.com/paper/2407.10230/) (2024)
- [Minimum-Risk Recalibration of Classifiers](https://mlsanity.com/paper/2305.10886/) (2023)
- [Beyond Probability Partitions: Calibrating Neural Networks with Semantic Aware Grouping](https://mlsanity.com/paper/2306.04985/) (2023)
- [SimCalib: Graph Neural Network Calibration based on Similarity between Nodes](https://mlsanity.com/paper/2312.11858/) (2023)
- [Revisiting Confidence Estimation: Towards Reliable Failure Prediction](https://mlsanity.com/paper/2403.02886/) (2024)
- [Taking a Step Back with KCal: Multi-Class Kernel-Based Calibration for Deep Neural Networks](https://mlsanity.com/paper/2202.07679/) (2022)
- [Accurate and Scalable Estimation of Epistemic Uncertainty for Graph Neural Networks](https://mlsanity.com/paper/2401.03350/) (2024)
- [Exploring Covariate and Concept Shift for Detection and Calibration of Out-of-Distribution Data](https://mlsanity.com/paper/2110.15231/) (2021)
- [Approaching Test Time Augmentation in the Context of Uncertainty Calibration for Deep Neural Networks](https://mlsanity.com/paper/2304.05104/) (2023)
- [Scaling of Class-wise Training Losses for Post-hoc Calibration](https://mlsanity.com/paper/2306.10989/) (2023)
- [Temperature-scaling surprisal estimates improve fit to human reading times -- but does it do so for the "right reasons"?](https://mlsanity.com/paper/2311.09325/) (2023)
- [C-Adapter: Adapting Deep Classifiers for Efficient Conformal Prediction Sets](https://mlsanity.com/paper/2410.09408/) (2024)
- [A calibration test for evaluating set-based epistemic uncertainty representations](https://mlsanity.com/paper/2502.16299/) (2025)
- [FedCal: Achieving Local and Global Calibration in Federated Learning via Aggregated Parameterized Scaler](https://mlsanity.com/paper/2405.15458/) (2024)
- [Understanding and Mitigating Under-Confidence in GNNs from the Final Layer](https://mlsanity.com/paper/2505.11335/) (2025)

## Abstract (arXiv)

Predicting calibrated confidence scores for multi-class deep networks is important for avoiding rare but costly mistakes. A common approach is to learn a post-hoc calibration function that transforms the output of the original network into calibrated confidence scores while maintaining the network's accuracy. However, previous post-hoc calibration techniques work only with simple calibration functions, potentially lacking sufficient representation to calibrate the complex function landscape of deep networks. In this work, we aim to learn general post-hoc calibration functions that can preserve the top-k predictions of any deep network. We call this family of functions intra order-preserving functions. We propose a new neural network architecture that represents a class of intra order-preserving functions by combining common neural network components. Additionally, we introduce order-invariant and diagonal sub-families, which can act as regularization for better generalization when the training data size is small. We show the effectiveness of the proposed method across a wide range of datasets and classifiers. Our method outperforms state-of-the-art post-hoc calibration methods, namely temperature scaling and Dirichlet calibration, in several evaluation metrics for the task.