mlsanityData to 7 Oct 2026

Robust Calibration with Multi-domain Temperature Scaling

arXiv 2206.02757 ↗First posted 6 Jun 2022cs.LG cs.AI stat.MLcited by 39

Not on the map. Why?

Abstract (arXiv)

Uncertainty quantification is essential for the reliable deployment of machine learning models to high-stakes application domains. Uncertainty quantification is all the more challenging when training distribution and test distribution are different, even the distribution shifts are mild. Despite the ubiquity of distribution shifts in real-world applications, existing uncertainty quantification approaches mainly study the in-distribution setting where the train and test distributions are the same. In this paper, we develop a systematic calibration model to handle distribution shifts by leveraging data from multiple domains. Our proposed method -- multi-domain temperature scaling -- uses the heterogeneity in the domains to improve calibration robustness under distribution shift. Through experiments on three benchmark data sets, we find our proposed method outperforms existing methods as measured on both in-distribution and out-of-distribution test sets.

Key facts. Cited by 39 papers in the map (data through 7 Oct 2026). Builds on 34 of its 54 references that are in the map. First posted on arXiv 6 Jun 2022.

Builds on

34 of 54 references.

References · 34

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2016Densely Connected Convolutional Networkscited by 6,691
  4. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  5. 2017On Calibration of Modern Neural Networkscited by 4,860
  6. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  7. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  8. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,857
  9. 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,540
  10. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,362
  11. 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,842
  12. 2020WILDS: A Benchmark of in-the-Wild Distribution Shiftscited by 1,366
22 more
  1. 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,274
  2. 2019AugMix: A Simple Data Processing Method to Improve Robustness and Uncertaintycited by 897
  3. 2019Big Transfer (BiT): General Visual Representation Learningcited by 766
  4. 2020Out-of-Distribution Generalization via Risk Extrapolation (REx)cited by 737
  5. 2019Conformal Prediction Under Covariate Shiftcited by 511
  6. 2019Using Self-Supervised Learning Can Improve Model Robustness and Uncertaintycited by 511
  7. 2021Revisiting the Calibration of Modern Neural Networkscited by 387
  8. 2020Google Landmarks Dataset v2 -- A Large-Scale Benchmark for Instance-Level Recognition and Retrievalcited by 305
  9. 2019On Mixup Training: Improved Calibration and Predictive Uncertainty for Deep Neural Networkscited by 304
  10. 2019Adversarial Examples Improve Image Recognitioncited by 283
  11. 2021Distribution-Free, Risk-Controlling Prediction Setscited by 278
  12. 2017Concrete Dropoutcited by 227
  13. 2021On Calibration and Out-of-domain Generalizationcited by 134
  14. 2021Predicting with Confidence on Unseen Distributionscited by 116
  15. 2021Distribution-free uncertainty quantification for classification under label shiftcited by 99
  16. 2020Robust Validation: Confident Predictions Even When Distributions Shiftcited by 82
  17. 2020SDE-Net: Equipping Deep Neural Networks with Uncertainty Estimatescited by 75
  18. 2019Calibration of Encoder Decoder Models for Neural Machine Translationcited by 67
  19. 2020Calibrated Prediction with Covariate Shift via Unsupervised Domain Adaptationcited by 43
  20. 2021PAC Prediction Sets Under Covariate Shiftcited by 42
  21. 2020Transferable Calibration with Lower Bias and Variance in Domain Adaptationcited by 41
  22. 2022T-Cal: An optimal test for the calibration of predictive modelscited by 24

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 11; 2024: 13; 2025: 9; 2026: 5.

39 papers.

Sort:

By area:

  1. 2023Federated Conformal Predictors for Distributed Uncertainty Quantificationcited by 46
  2. 2024Thermometer: Towards Universal Calibration for Large Language Modelscited by 38
  3. 2023Towards Calibrated Robust Fine-Tuning of Vision-Language Modelscited by 35
  4. 2022Calibrated Selective Classificationcited by 30
  5. 2025Restoring Calibration for Aligned Large Language Models: A Calibration-Aware Fine-Tuning Approachcited by 25
  6. 2024An Empirical Study Into What Matters for Calibrating Vision-Language Modelscited by 20
  7. 2024MixedNUTS: Training-Free Accuracy-Robustness Balance via Nonlinearly Mixed Classifierscited by 16
  8. 2024Open-Vocabulary Calibration for Fine-tuned CLIPcited by 15
  9. 2023Beyond In-Domain Scenarios: Robust Density-Aware Calibrationcited by 14
  10. 2024A Framework for Efficient Model Evaluation through Stratification, Sampling, and Estimationcited by 13
  11. 2024Few-Shot Recalibration of Language Modelscited by 8
  12. 2023Federated Uncertainty-Aware Aggregation for Fundus Diabetic Retinopathy Stagingcited by 7
27 more
  1. 2024Calibration Attacks: A Comprehensive Study of Adversarial Attacks on Model Confidencecited by 7
  2. 2024Predicting the Performance of Foundation Models via Agreement-on-the-Linecited by 7
  3. 2026BaseCal: Unsupervised Confidence Calibration via Base Model Signalscited by 7
  4. 2023MaxEnt Loss: Constrained Maximum Entropy for Calibration under Out-of-Distribution Shiftcited by 6
  5. 2024FedCal: Achieving Local and Global Calibration in Federated Learning via Aggregated Parameterized Scalercited by 6
  6. 2024Selective Learning: Towards Robust Calibration with Dynamic Regularizationcited by 5
  7. 2025T-CIL: Temperature Scaling using Adversarial Perturbation for Calibration in Class-Incremental Learningcited by 4
  8. 2025Where are we with calibration under dataset shift in image classification?cited by 4
  9. 2023Model Calibration in Dense Classification with Adaptive Label Perturbationcited by 3
  10. 2025Towards Unveiling Predictive Uncertainty Vulnerabilities in the Context of the Right to Be Forgottencited by 3
  11. 2023Generalized Uncertainty of Deep Neural Networks: Taxonomy and Applicationscited by 2
  12. 2023Confidence Calibration for Systems with Cascaded Predictive Modulescited by 2
  13. 2024Consistency-Guided Temperature Scaling Using Style and Content Information for Out-of-Domain Calibrationcited by 2
  14. 2025Improving Perturbation-based Explanations by Understanding the Role of Uncertainty Calibrationcited by 2
  15. 2026Quantile Adaptive Temperature Scaling for Confidence Calibrationcited by 2
  16. 2023IW-GAE: Importance Weighted Group Accuracy Estimation for Improved Calibration and Model Selection in Unsupervised Domain Adaptationcited by 1
  17. 2024Domain-adaptive and Subgroup-specific Cascaded Temperature Regression for Out-of-distribution Calibrationcited by 1
  18. 2024Improving self-training under distribution shifts via anchored confidence with theoretical guaranteescited by 1
  19. 2025Conformal Prediction Adaptive to Unknown Subpopulation Shiftscited by 1
  20. 2025Target-Agnostic Calibration under Distribution Shift with Frequency-Aware Gradient Rectificationcited by 1
  21. 2025Towards Reliable Test-Time Adaptation: Style Invariance as a Correctness Likelihoodcited by 1
  22. 2026DA-Cal: Towards Cross-Domain Calibration in Semantic Segmentationcited by 1
  23. 2023PseudoCal: A Source-Free Approach to Unsupervised Uncertainty Calibration in Domain Adaptationcited by 0
  24. 2023Unified Uncertainty Calibrationcited by 0
  25. 2025Why Uncertainty Calibration Matters for Reliable Perturbation-based Explanationscited by 0
  26. 2026Fine-grained Approaches for Confidence Calibration of LLMs in Automated Code Revisioncited by 0
  27. 2026Beyond Post-Hoc Temperature Scaling: Bilevel Optimization for LLM Calibrationcited by 0