# On Temperature Scaling and Conformal Prediction of Deep Classifiers

arXiv: https://arxiv.org/abs/2402.05806 · HTML: https://mlsanity.com/paper/2402.05806/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 43 references that are in the map.
- First posted on arXiv 8 Feb 2024.

## Builds on (most cited first, 23 of 23)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks](https://mlsanity.com/paper/1706.02690/) (2017)
- [A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantification](https://mlsanity.com/paper/2107.07511/) (2021)
- [Conformal Prediction Under Covariate Shift](https://mlsanity.com/paper/1904.06019/) (2019)
- [Measuring Calibration in Deep Learning](https://mlsanity.com/paper/1904.01685/) (2019)
- [A Survey of Deep Learning Techniques for Autonomous Driving](https://mlsanity.com/paper/1910.07738/) (2019)
- [Classification with Valid and Adaptive Coverage](https://mlsanity.com/paper/2006.02544/) (2020)
- [Uncertainty Sets for Image Classifiers using Conformal Prediction](https://mlsanity.com/paper/2009.14193/) (2020)
- [Conformal prediction beyond exchangeability](https://mlsanity.com/paper/2202.13415/) (2022)
- [Least Ambiguous Set-Valued Classifiers with Bounded Error Levels](https://mlsanity.com/paper/1609.00451/) (2016)
- [Mitigating Neural Network Overconfidence with Logit Normalization](https://mlsanity.com/paper/2205.09310/) (2022)
- [Conformal Prediction With Conditional Guarantees](https://mlsanity.com/paper/2305.12616/) (2023)
- [Learning Optimal Conformal Classifiers](https://mlsanity.com/paper/2110.09192/) (2021)
- [Class-Conditional Conformal Prediction with Many Classes](https://mlsanity.com/paper/2306.09335/) (2023)
- [Local Temperature Scaling for Probability Calibration](https://mlsanity.com/paper/2008.05105/) (2020)
- [Distribution-free binary classification: prediction sets, confidence intervals and calibration](https://mlsanity.com/paper/2006.10564/) (2020)
- [Federated Conformal Predictors for Distributed Uncertainty Quantification](https://mlsanity.com/paper/2305.17564/) (2023)
- [Bin-wise Temperature Scaling (BTS): Improvement in Confidence Calibration Performance through Simple Scaling Techniques](https://mlsanity.com/paper/1908.11528/) (2019)
- [Does confidence calibration improve conformal prediction?](https://mlsanity.com/paper/2402.04344/) (2024)
- [Estimating Test Performance for AI Medical Devices under Distribution Shift with Conformal Prediction](https://mlsanity.com/paper/2207.05796/) (2022)

## Built on by (most cited first, 13 of 13)

- [Boosted Conformal Prediction Intervals](https://mlsanity.com/paper/2406.07449/) (2024)
- [Parametric Scaling Law of Tuning Bias in Conformal Prediction](https://mlsanity.com/paper/2502.03023/) (2025)
- [ConfIC-RCA: Statistically Grounded Efficient Estimation of Segmentation Quality](https://mlsanity.com/paper/2503.04522/) (2025)
- [Sample Margin-Aware Recalibration of Temperature Scaling](https://mlsanity.com/paper/2506.23492/) (2025)
- [Efficient Conformal Prediction for Regression Models under Label Noise](https://mlsanity.com/paper/2509.15120/) (2025)
- [The Well-Tempered Classifier: Some Elementary Properties of Temperature Scaling](https://mlsanity.com/paper/2602.14862/) (2026)
- [Aerial Image Classification in Scarce and Unconstrained Environments via Conformal Prediction](https://mlsanity.com/paper/2504.17655/) (2025)
- [Class conditional conformal prediction for multiple inputs by p-value aggregation](https://mlsanity.com/paper/2507.07150/) (2025)
- [Enhancing Conformal Prediction via Class Similarity](https://mlsanity.com/paper/2511.19359/) (2025)
- [Conformal Correction for Efficiency May be at Odds with Entropy](https://mlsanity.com/paper/2512.02704/) (2025)
- [Questioning the Coverage-Length Metric in Conformal Prediction: When Shorter Intervals Are Not Better](https://mlsanity.com/paper/2601.21455/) (2026)
- [Robust Conditional Conformal Prediction via Branched Normalizing Flow](https://mlsanity.com/paper/2605.01868/) (2026)
- [Missingness-Aware Conformal Prediction Under Cross-Hospital Distribution Shift](https://mlsanity.com/paper/2609.30781/) (2026)

## Abstract (arXiv)

In many classification applications, the prediction of a deep neural network (DNN) based classifier needs to be accompanied by some confidence indication. Two popular approaches for that aim are: 1) Calibration: modifies the classifier's softmax values such that the maximal value better estimates the correctness probability; and 2) Conformal Prediction (CP): produces a prediction set of candidate labels that contains the true label with a user-specified probability, guaranteeing marginal coverage but not, e.g., per class coverage. In practice, both types of indications are desirable, yet, so far the interplay between them has not been investigated. Focusing on the ubiquitous Temperature Scaling (TS) calibration, we start this paper with an extensive empirical study of its effect on prominent CP methods. We show that while TS calibration improves the class-conditional coverage of adaptive CP methods, surprisingly, it negatively affects their prediction set sizes. Motivated by this behavior, we explore the effect of TS on CP beyond its calibration application and reveal an intriguing trend under which it allows to trade prediction set size and conditional coverage of adaptive CP methods. Then, we establish a mathematical theory that explains the entire non-monotonic trend. Finally, based on our experiments and theory, we offer simple guidelines for practitioners to effectively combine adaptive CP with calibration, aligned with user-defined goals.