Too Sharp, Too Sure: When Calibration Follows Curvature
Not on the map. Why?
Abstract (arXiv)
Modern neural networks can achieve high accuracy while remaining poorly calibrated, producing confidence estimates that do not match empirical correctness. Yet calibration is often treated as a post-hoc attribute. We take a different perspective: we study calibration as a training-time phenomenon on small vision tasks, and ask whether calibrated solutions can be obtained reliably by intervening on the training procedure. We identify a tight coupling between calibration, curvature, and margins during training of deep networks under multiple gradient-based methods. Empirically, Expected Calibration Error (ECE) closely tracks curvature-based sharpness throughout optimization. Mathematically, we show that both ECE and Gauss--Newton curvature are controlled, up to problem-specific constants, by the same margin-dependent exponential tail functional along the trajectory. Guided by this mechanism, we introduce a margin-aware training objective that explicitly targets robust-margin tails and local smoothness, yielding improved out-of-sample calibration across optimizers without sacrificing accuracy.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 42 of its 56 references that are in the map. First posted on arXiv 22 Apr 2026.
Builds on
42 of 56 references.
References · 42
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2017On Calibration of Modern Neural Networkscited by 4,960
- 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,517
- 2020Sharpness-Aware Minimization for Efficiently Improving Generalizationcited by 1,417
- 2016On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minimacited by 1,352
- 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,287
- 2019When Does Label Smoothing Help?cited by 988
- 2017Spectrally-normalized margin bounds for neural networkscited by 686
- 2017The Implicit Bias of Gradient Descent on Separable Datacited by 655
- 2017Exploring Generalization in Deep Learningcited by 600
- 2019Fantastic Generalization Measures and Where to Find Themcited by 513
- 2017Sharp Minima Can Generalize For Deep Netscited by 466
30 more
- 2017Regularizing Neural Networks by Penalizing Confident Output Distributionscited by 464
- 2021Gradient Descent on Neural Networks Typically Occurs at the Edge of Stabilitycited by 427
- 2019Beyond temperature scaling: Obtaining well-calibrated multiclass probabilities with Dirichlet calibrationcited by 364
- 2016Entropy-SGD: Biasing Gradient Descent Into Wide Valleyscited by 359
- 2020Calibrating Deep Neural Networks using Focal Losscited by 357
- 2019On Mixup Training: Improved Calibration and Predictive Uncertainty for Deep Neural Networkscited by 305
- 2017Train longer, generalize better: closing the generalization gap in large batch training of neural networkscited by 279
- 2020Mix-n-Match: Ensemble and Compositional Methods for Uncertainty Calibration in Deep Learningcited by 190
- 2022Adaptive Gradient Methods at the Edge of Stabilitycited by 101
- 2017Fisher-Rao Metric, Geometry, and Complexity of Neural Networkscited by 100
- 2021The Devil is in the Margin: Margin-based Label Smoothing for Network Calibrationcited by 91
- 2018On the Relation Between the Sharpest Directions of DNN Loss and the SGD Step Lengthcited by 88
- 2018A Walk with SGDcited by 78
- 2020Regularizing Neural Networks via Adversarial Model Perturbationcited by 78
- 2019Normalized Flat Minima: Exploring Scale Invariant Definition of Flat Minima for Neural Networks using PAC-Bayesian Analysiscited by 59
- 2021Relating Adversarially Robust Generalization to Flat Minimacited by 57
- 2019Deterministic PAC-Bayesian generalization bounds for deep networks via generalizing noise-resiliencecited by 51
- 2020Rethinking Parameter Counting in Deep Models: Effective Dimensionality Revisitedcited by 51
- 2022SAM as an Optimal Relaxation of Bayescited by 41
- 2024Does SGD really happen in tiny subspaces?cited by 36
- 2024Edge of Stochastic Stability: Revisiting the Edge of Stability for SGDcited by 30
- 2021Meta-Calibration: Learning of Model Calibration Using Differentiable Expected Calibration Errorcited by 29
- 2020Improving Calibration through the Relationship with Adversarial Robustnesscited by 26
- 2024Sharpness-Aware Minimization Efficiently Selects Flatter Minima Late in Trainingcited by 24
- 2022The Calibration Generalization Gapcited by 18
- 2025Rethinking Early Stopping: Refine, Then Calibratecited by 17
- 2023Outliers with Opposing Signals Have an Outsized Effect on Neural Network Optimizationcited by 13
- 2025Benefits of Early Stopping in Gradient Descent for Overparameterized Logistic Regressioncited by 9
- 2025Optimal and Provable Calibration in High-Dimensional Binary Classification: Angular Calibration and Platt Scalingcited by 2
- 2025Towards Understanding The Calibration Benefits of Sharpness-Aware Minimizationcited by 2
Built on by
Citing papers per year: 2026: 3.
3 papers.