mlsanityData to 7 Oct 2026

Towards Understanding The Calibration Benefits of Sharpness-Aware Minimization

arXiv 2505.23866 ↗First posted 29 May 2025cs.LG cs.AIcited by 2

Not on the map. Why?

Abstract (arXiv)

Deep neural networks have been increasingly used in safety-critical applications such as medical diagnosis and autonomous driving. However, many studies suggest that they are prone to being poorly calibrated and have a propensity for overconfidence, which may have disastrous consequences. In this paper, unlike standard training such as stochastic gradient descent, we show that the recently proposed sharpness-aware minimization (SAM) counteracts this tendency towards overconfidence. The theoretical analysis suggests that SAM allows us to learn models that are already well-calibrated by implicitly maximizing the entropy of the predictive distribution. Inspired by this finding, we further propose a variant of SAM, coined as CSAM, to ameliorate model calibration. Extensive experiments on various datasets, including ImageNet-1K, demonstrate the benefits of SAM in reducing calibration error. Meanwhile, CSAM performs even better than SAM and consistently achieves lower calibration error than other approaches

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 75 of its 96 references that are in the map. First posted on arXiv 29 May 2025.

Builds on

75 of 96 references.

References · 75

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2016Densely Connected Convolutional Networkscited by 6,691
  4. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  5. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  6. 2017On Calibration of Modern Neural Networkscited by 4,860
  7. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  8. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  9. 2016Wide Residual Networkscited by 3,057
  10. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,857
  11. 2021MLP-Mixer: An all-MLP Architecture for Visioncited by 1,441
  12. 2020Sharpness-Aware Minimization for Efficiently Improving Generalizationcited by 1,404
38 more
  1. 2016On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minimacited by 1,337
  2. 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,274
  3. 2018Averaging Weights Leads to Wider Optima and Better Generalizationcited by 1,194
  4. 2019When Does Label Smoothing Help?cited by 985
  5. 2014Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Imagescited by 774
  6. 2015Train faster, generalize better: Stability of stochastic gradient descentcited by 547
  7. 2017Sharp Minima Can Generalize For Deep Netscited by 462
  8. 2017Regularizing Neural Networks by Penalizing Confident Output Distributionscited by 461
  9. 2021Revisiting the Calibration of Modern Neural Networkscited by 387
  10. 2020BatchEnsemble: An Alternative Approach to Efficient Ensemble and Lifelong Learningcited by 368
  11. 2019Beyond temperature scaling: Obtaining well-calibrated multiclass probabilities with Dirichlet calibrationcited by 359
  12. 2016Entropy-SGD: Biasing Gradient Descent Into Wide Valleyscited by 357
  13. 2020Calibrating Deep Neural Networks using Focal Losscited by 353
  14. 2017Snapshot Ensembles: Train 1, get M for freecited by 349
  15. 2019Lookahead Optimizer: k steps forward, 1 step backcited by 333
  16. 2019On Mixup Training: Improved Calibration and Predictive Uncertainty for Deep Neural Networkscited by 304
  17. 2021ASAM: Adaptive Sharpness-Aware Minimization for Scale-Invariant Learning of Deep Neural Networkscited by 293
  18. 2021When Vision Transformers Outperform ResNets without Pre-training or Strong Data Augmentationscited by 259
  19. 2020Training independent subnetworks for robust predictioncited by 177
  20. 2020Efficient and Scalable Bayesian Neural Nets with Rank-1 Factorscited by 171
  21. 2016Deep Pyramidal Residual Networkscited by 163
  22. 2022Towards Understanding Sharpness-Aware Minimizationcited by 145
  23. 2021Efficient Sharpness-aware Minimization for Improved Training of Neural Networkscited by 137
  24. 2022Towards Efficient and Scalable Sharpness-Aware Minimizationcited by 129
  25. 2022Penalizing Gradient Norm for Efficiently Improving Generalization in Deep Learningcited by 122
  26. 2020Confidence-Aware Learning for Deep Neural Networkscited by 113
  27. 2021Sharpness-Aware Minimization Improves Language Model Generalizationcited by 108
  28. 2023Recent Advancements in End-to-End Autonomous Driving using Deep Learning: A Surveycited by 101
  29. 2021The Devil is in the Margin: Margin-based Label Smoothing for Network Calibrationcited by 91
  30. 2023A Modern Look at the Relationship between Sharpness and Generalizationcited by 91
  31. 2022Fisher SAM: Information Geometry and Sharpness Aware Minimisationcited by 84
  32. 2020On Mixup Regularizationcited by 82
  33. 2020Regularizing Neural Networks via Adversarial Model Perturbationcited by 78
  34. 2022When Do Flat Minima Optimizers Work?cited by 78
  35. 2023Calibration in Deep Learning: A Survey of the State-of-the-Artcited by 72
  36. 2022Make Sharpness-Aware Minimization Stronger: A Sparsified Perturbation Approachcited by 70
  37. 2022How Does Sharpness-Aware Minimization Minimize Sharpness?cited by 64
  38. 2022The Dynamics of Sharpness-Aware Minimization: Bouncing Across Ravines and Drifting Towards Wide Minimacited by 51

50 most cited of 75 All 75 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Too Sharp, Too Sure: When Calibration Follows Curvaturecited by 2
  2. 2026Improving Calibration in Test-Time Prompt Tuning for Vision-Language Models via Data-Free Flatness-Aware Prompt Pretrainingcited by 2