# Robust Calibration with Multi-domain Temperature Scaling

arXiv: https://arxiv.org/abs/2206.02757 · HTML: https://mlsanity.com/paper/2206.02757/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 54 references that are in the map.
- First posted on arXiv 6 Jun 2022.

## Builds on (most cited first, 30 of 34)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [Aggregated Residual Transformations for Deep Neural Networks](https://mlsanity.com/paper/1611.05431/) (2016)
- [A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks](https://mlsanity.com/paper/1610.02136/) (2016)
- [The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization](https://mlsanity.com/paper/2006.16241/) (2020)
- [WILDS: A Benchmark of in-the-Wild Distribution Shifts](https://mlsanity.com/paper/2012.07421/) (2020)
- [Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift](https://mlsanity.com/paper/1906.02530/) (2019)
- [AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty](https://mlsanity.com/paper/1912.02781/) (2019)
- [Big Transfer (BiT): General Visual Representation Learning](https://mlsanity.com/paper/1912.11370/) (2019)
- [Out-of-Distribution Generalization via Risk Extrapolation (REx)](https://mlsanity.com/paper/2003.00688/) (2020)
- [Conformal Prediction Under Covariate Shift](https://mlsanity.com/paper/1904.06019/) (2019)
- [Using Self-Supervised Learning Can Improve Model Robustness and Uncertainty](https://mlsanity.com/paper/1906.12340/) (2019)
- [Revisiting the Calibration of Modern Neural Networks](https://mlsanity.com/paper/2106.07998/) (2021)
- [Google Landmarks Dataset v2 -- A Large-Scale Benchmark for Instance-Level Recognition and Retrieval](https://mlsanity.com/paper/2004.01804/) (2020)
- [On Mixup Training: Improved Calibration and Predictive Uncertainty for Deep Neural Networks](https://mlsanity.com/paper/1905.11001/) (2019)
- [Distribution-Free, Risk-Controlling Prediction Sets](https://mlsanity.com/paper/2101.02703/) (2021)
- [Adversarial Examples Improve Image Recognition](https://mlsanity.com/paper/1911.09665/) (2019)
- [Concrete Dropout](https://mlsanity.com/paper/1705.07832/) (2017)
- [On Calibration and Out-of-domain Generalization](https://mlsanity.com/paper/2102.10395/) (2021)
- [Predicting with Confidence on Unseen Distributions](https://mlsanity.com/paper/2107.03315/) (2021)
- [Distribution-free uncertainty quantification for classification under label shift](https://mlsanity.com/paper/2103.03323/) (2021)
- [Robust Validation: Confident Predictions Even When Distributions Shift](https://mlsanity.com/paper/2008.04267/) (2020)
- [SDE-Net: Equipping Deep Neural Networks with Uncertainty Estimates](https://mlsanity.com/paper/2008.10546/) (2020)
- [Calibration of Encoder Decoder Models for Neural Machine Translation](https://mlsanity.com/paper/1903.00802/) (2019)

## Built on by (most cited first, 30 of 39)

- [Federated Conformal Predictors for Distributed Uncertainty Quantification](https://mlsanity.com/paper/2305.17564/) (2023)
- [Thermometer: Towards Universal Calibration for Large Language Models](https://mlsanity.com/paper/2403.08819/) (2024)
- [Towards Calibrated Robust Fine-Tuning of Vision-Language Models](https://mlsanity.com/paper/2311.01723/) (2023)
- [Calibrated Selective Classification](https://mlsanity.com/paper/2208.12084/) (2022)
- [Restoring Calibration for Aligned Large Language Models: A Calibration-Aware Fine-Tuning Approach](https://mlsanity.com/paper/2505.01997/) (2025)
- [An Empirical Study Into What Matters for Calibrating Vision-Language Models](https://mlsanity.com/paper/2402.07417/) (2024)
- [MixedNUTS: Training-Free Accuracy-Robustness Balance via Nonlinearly Mixed Classifiers](https://mlsanity.com/paper/2402.02263/) (2024)
- [Open-Vocabulary Calibration for Fine-tuned CLIP](https://mlsanity.com/paper/2402.04655/) (2024)
- [Beyond In-Domain Scenarios: Robust Density-Aware Calibration](https://mlsanity.com/paper/2302.05118/) (2023)
- [A Framework for Efficient Model Evaluation through Stratification, Sampling, and Estimation](https://mlsanity.com/paper/2406.07320/) (2024)
- [Few-Shot Recalibration of Language Models](https://mlsanity.com/paper/2403.18286/) (2024)
- [BaseCal: Unsupervised Confidence Calibration via Base Model Signals](https://mlsanity.com/paper/2601.03042/) (2026)
- [Federated Uncertainty-Aware Aggregation for Fundus Diabetic Retinopathy Staging](https://mlsanity.com/paper/2303.13033/) (2023)
- [Calibration Attacks: A Comprehensive Study of Adversarial Attacks on Model Confidence](https://mlsanity.com/paper/2401.02718/) (2024)
- [Predicting the Performance of Foundation Models via Agreement-on-the-Line](https://mlsanity.com/paper/2404.01542/) (2024)
- [MaxEnt Loss: Constrained Maximum Entropy for Calibration under Out-of-Distribution Shift](https://mlsanity.com/paper/2310.17159/) (2023)
- [FedCal: Achieving Local and Global Calibration in Federated Learning via Aggregated Parameterized Scaler](https://mlsanity.com/paper/2405.15458/) (2024)
- [Selective Learning: Towards Robust Calibration with Dynamic Regularization](https://mlsanity.com/paper/2402.08384/) (2024)
- [T-CIL: Temperature Scaling using Adversarial Perturbation for Calibration in Class-Incremental Learning](https://mlsanity.com/paper/2503.22163/) (2025)
- [Where are we with calibration under dataset shift in image classification?](https://mlsanity.com/paper/2507.07780/) (2025)
- [Model Calibration in Dense Classification with Adaptive Label Perturbation](https://mlsanity.com/paper/2307.13539/) (2023)
- [Towards Unveiling Predictive Uncertainty Vulnerabilities in the Context of the Right to Be Forgotten](https://mlsanity.com/paper/2508.07458/) (2025)
- [Improving Perturbation-based Explanations by Understanding the Role of Uncertainty Calibration](https://mlsanity.com/paper/2511.10439/) (2025)
- [Generalized Uncertainty of Deep Neural Networks: Taxonomy and Applications](https://mlsanity.com/paper/2302.01440/) (2023)
- [Confidence Calibration for Systems with Cascaded Predictive Modules](https://mlsanity.com/paper/2309.12510/) (2023)
- [Consistency-Guided Temperature Scaling Using Style and Content Information for Out-of-Domain Calibration](https://mlsanity.com/paper/2402.15019/) (2024)
- [Improving self-training under distribution shifts via anchored confidence with theoretical guarantees](https://mlsanity.com/paper/2411.00586/) (2024)
- [Conformal Prediction Adaptive to Unknown Subpopulation Shifts](https://mlsanity.com/paper/2506.05583/) (2025)
- [Quantile Adaptive Temperature Scaling for Confidence Calibration](https://mlsanity.com/paper/2606.21749/) (2026)
- [IW-GAE: Importance Weighted Group Accuracy Estimation for Improved Calibration and Model Selection in Unsupervised Domain Adaptation](https://mlsanity.com/paper/2310.10611/) (2023)

## Abstract (arXiv)

Uncertainty quantification is essential for the reliable deployment of machine learning models to high-stakes application domains. Uncertainty quantification is all the more challenging when training distribution and test distribution are different, even the distribution shifts are mild. Despite the ubiquity of distribution shifts in real-world applications, existing uncertainty quantification approaches mainly study the in-distribution setting where the train and test distributions are the same. In this paper, we develop a systematic calibration model to handle distribution shifts by leveraging data from multiple domains. Our proposed method -- multi-domain temperature scaling -- uses the heterogeneity in the domains to improve calibration robustness under distribution shift. Through experiments on three benchmark data sets, we find our proposed method outperforms existing methods as measured on both in-distribution and out-of-distribution test sets.