mlsanityData to 7 Oct 2026

Meta-Calibration: Learning of Model Calibration Using Differentiable Expected Calibration Error

arXiv 2106.09613 ↗First posted 17 Jun 2021cs.LG stat.MLcited by 28

Not on the map. Why?

Abstract (arXiv)

Calibration of neural networks is a topical problem that is becoming more and more important as neural networks increasingly underpin real-world applications. The problem is especially noticeable when using modern neural networks, for which there is a significant difference between the confidence of the model and the probability of correct prediction. Various strategies have been proposed to improve calibration, yet accurate calibration remains challenging. We propose a novel framework with two contributions: introducing a new differentiable surrogate for expected calibration error (DECE) that allows calibration quality to be directly optimised, and a meta-learning framework that uses DECE to optimise for validation set calibration with respect to model hyper-parameters. The results show that we achieve competitive performance with existing calibration approaches. Our framework opens up a new avenue and toolset for tackling calibration, which we believe will inspire further work on this important challenge.

Key facts. Cited by 28 papers in the map (data through 7 Oct 2026). Builds on 37 of its 79 references that are in the map. First posted on arXiv 17 Jun 2021.

Builds on

37 of 79 references.

References · 37

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  4. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  5. 2017On Calibration of Modern Neural Networkscited by 4,860
  6. 2016Wide Residual Networkscited by 3,057
  7. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,857
  8. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,362
  9. 2017Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networkscited by 1,325
  10. 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,274
  11. 2016End to End Learning for Self-Driving Carscited by 1,000
  12. 2019When Does Label Smoothing Help?cited by 985
25 more
  1. 2020Meta-Learning in Neural Networks: A Surveycited by 944
  2. 2013Network In Networkcited by 600
  3. 2019Measuring Calibration in Deep Learningcited by 443
  4. 2017Training Confidence-calibrated Classifiers for Detecting Out-of-Distribution Samplescited by 389
  5. 2021Revisiting the Calibration of Modern Neural Networkscited by 387
  6. 2019Beyond temperature scaling: Obtaining well-calibrated multiclass probabilities with Dirichlet calibrationcited by 359
  7. 2020Calibrating Deep Neural Networks using Focal Losscited by 353
  8. 2019Verified Uncertainty Calibrationcited by 307
  9. 2015Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networkscited by 305
  10. 2019Optimizing Millions of Hyperparameters by Implicit Differentiationcited by 303
  11. 2018Dataset Distillationcited by 224
  12. 2020Fast Differentiable Sorting and Rankingcited by 196
  13. 2020Adaptive Risk Minimization: Learning to Adapt to Domain Shiftcited by 183
  14. 2019Evaluating model calibration in classificationcited by 165
  15. 2019Feature-Critic Networks for Heterogeneous Domain Generalizationcited by 130
  16. 2020Improving model calibration with accuracy versus uncertainty optimizationcited by 123
  17. 2020Flexible Dataset Distillation: Learn Labels Instead of Imagescited by 91
  18. 2019Calibration tests in multi-class classification: A unifying frameworkcited by 79
  19. 2018Deep Neural Decision Treescited by 72
  20. 2015Scalable Gradient-Based Tuning of Continuous Regularization Hyperparameterscited by 71
  21. 2020Post-hoc Uncertainty Calibration for Domain Drift Scenarioscited by 61
  22. 2021MetaXL: Meta Representation Transformation for Low-resource Cross-lingual Learningcited by 21
  23. 2019A neural network-based framework for financial model calibrationcited by 8
  24. 2021MetaLabelNet: Learning to Generate Soft-Labels from Noisy-Labelscited by 5
  25. 2019Learning to Impute: A General Framework for Semi-supervised Learningcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 8; 2024: 7; 2025: 8; 2026: 3.

28 papers.

Sort:

By area:

  1. 2023Calibration in Deep Learning: A Survey of the State-of-the-Artcited by 72
  2. 2022Adaptive Temperature Scaling for Robust Calibration of Deep Neural Networkscited by 24
  3. 2023Calibration by Distribution Matching: Trainable Kernel Calibration Metricscited by 20
  4. 2022Calibrating Deep Neural Networks using Explicit Regularisation and Dynamic Data Pruningcited by 18
  5. 2024Are foundation models for computer vision good conformal predictors?cited by 12
  6. 2023On the Overconfidence Problem in Semantic 3D Mappingcited by 7
  7. 2024Average Calibration Error: A Differentiable Loss for Improved Reliability in Image Segmentationcited by 6
  8. 2024Calibrating Bayesian Learning via Regularization, Confidence Minimization, and Selective Inferencecited by 6
  9. 2024Towards Certification of Uncertainty Calibration under Adversarial Attackscited by 5
  10. 2025Uncertainty Weighted Gradients for Model Calibrationcited by 5
  11. 2025Refine and Align: Confidence Calibration through Multi-Agent Interaction in VQAcited by 4
  12. 2023Calibration-Aware Bayesian Learningcited by 3
16 more
  1. 2025Average Calibration Losses for Reliable Uncertainty in Medical Image Segmentationcited by 3
  2. 2025Distilling Calibration via Conformalized Credal Inferencecited by 2
  3. 2025h-calibration: Rethinking Classifier Recalibration with Probabilistic Error-Bounded Objectivecited by 2
  4. 2026Too Sharp, Too Sure: When Calibration Follows Curvaturecited by 2
  5. 2023Towards Unbiased Calibration using Meta-Regularizationcited by 1
  6. 2023AUGCAL: Improving Sim2Real Adaptation by Uncertainty Calibration on Augmented Synthetic Imagescited by 1
  7. 2024Rethinking Meta-Learning from a Learning Lenscited by 1
  8. 2024The Craft of Selective Prediction: Towards Reliable Case Outcome Classification -- An Empirical Study on European Court of Human Rights Casescited by 1
  9. 2025Enhancing Diffusion Model Guidance through Calibration and Regularizationcited by 1
  10. 2023AwesomeMeta+: A Mixed-Prototyping Meta-Learning System Supporting AI Application Design Anywherecited by 0
  11. 2023A Saliency-based Clustering Framework for Identifying Aberrant Predictionscited by 0
  12. 2024Theory-inspired Label Shift Adaptation via Aligned Distribution Mixturecited by 0
  13. 2025Revisiting Reweighted Risk for Calibration: AURC, Focal, and Inverse Focal Losscited by 0
  14. 2025Bag of Coins: A Statistical Probe into Neural Confidence Structurescited by 0
  15. 2026Socrates Loss: Unifying Confidence Calibration and Classification by Leveraging the Unknowncited by 0
  16. 2026Expectation Consistency Loss: Rethink Confidence Calibration under Covariate Shiftcited by 0