mlsanityData to 7 Oct 2026

A Geometric Perspective towards Neural Calibration via Sensitivity Decomposition

arXiv 2110.14577 ↗First posted 27 Oct 2021cs.CV cs.LGcited by 19

Not on the map. Why?

Abstract (arXiv)

It is well known that vision classification models suffer from poor calibration in the face of data distribution shifts. In this paper, we take a geometric approach to this problem. We propose Geometric Sensitivity Decomposition (GSD) which decomposes the norm of a sample feature embedding and the angular similarity to a target classifier into an instance-dependent and an instance-independent component. The instance-dependent component captures the sensitive information about changes in the input while the instance-independent component represents the insensitive information serving solely to minimize the loss on the training dataset. Inspired by the decomposition, we analytically derive a simple extension to current softmax-linear models, which learns to disentangle the two components during training. On several common vision models, the disentangled model outperforms other calibration methods on standard calibration metrics in the face of out-of-distribution (OOD) data and corruption with significantly less complexity. Specifically, we surpass the current state of the art by 30.8% relative improvement on corrupted CIFAR100 in Expected Calibration Error. Code available at https://github.com/GT-RIPL/Geometric-Sensitivity-Decomposition.git.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). Builds on 16 of its 30 references that are in the map. First posted on arXiv 27 Oct 2021.

Builds on

16 of 30 references.

References · 16

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2016Densely Connected Convolutional Networkscited by 6,691
  3. 2017On Calibration of Modern Neural Networkscited by 4,860
  4. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  5. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  6. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,857
  7. 2017What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?cited by 2,335
  8. 2017Improved Training of Wasserstein GANscited by 2,223
  9. 2018Spectral Normalization for Generative Adversarial Networkscited by 1,618
  10. 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,274
  11. 2017The Implicit Bias of Gradient Descent on Separable Datacited by 649
  12. 2020Simple and Principled Uncertainty Estimation with Deterministic Deep Learning via Distance Awarenesscited by 389
4 more
  1. 2016Large-Margin Softmax Loss for Convolutional Neural Networkscited by 296
  2. 2018Decoupled Networkscited by 45
  3. 2019Angular Visual Hardnesscited by 29
  4. 2020Deep Feature Space: A Geometrical Perspectivecited by 10

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 9; 2023: 7; 2024: 1; 2025: 1.

19 papers.

Sort:

  1. 2022Better Uncertainty Calibration via Proper Scores for Classification and Beyondcited by 65
  2. 2023Trainable Projected Gradient Method for Robust Fine-tuningcited by 37
  3. 2022Exploring Predictive Uncertainty and Calibration in NLP: A Study on the Impact of Method & Data Scarcitycited by 24
  4. 2022Open-Set Semi-Supervised Object Detectioncited by 17
  5. 2023Calibrating Multimodal Learningcited by 17
  6. 2023Consistent and Asymptotically Unbiased Estimation of Proper Calibration Errorscited by 10
  7. 2021Exploring Covariate and Concept Shift for Detection and Calibration of Out-of-Distribution Datacited by 7
  8. 2023Approaching Test Time Augmentation in the Context of Uncertainty Calibration for Deep Neural Networkscited by 6
  9. 2022Neural Clamping: Joint Input Perturbation and Temperature Scaling for Neural Network Calibrationcited by 4
  10. 2023Information-theoretic Analysis of Test Data Sensitivity in Uncertaintycited by 3
  11. 2024Optimizing Estimators of Squared Calibration Errors in Classificationcited by 3
  12. 2022Layer-Wise Partitioning and Merging for Efficient and Scalable Deep Learningcited by 2
7 more
  1. 2022NCTV: Neural Clamping Toolkit and Visualization for Neural Network Calibrationcited by 2
  2. 2025h-calibration: Rethinking Classifier Recalibration with Probabilistic Error-Bounded Objectivecited by 2
  3. 2023Adaptive Calibrator Ensemble for Model Calibration under Distribution Shiftcited by 1
  4. 2023A Theoretical and Practical Framework for Evaluating Uncertainty Calibration in Object Detectioncited by 1
  5. 2022Excess risk analysis for epistemic uncertainty with application to variational inferencecited by 0
  6. 2022Explanation-based Counterfactual Retraining(XCR): A Calibration Method for Black-box Modelscited by 0
  7. 2022MixBoost: Improving the Robustness of Deep Neural Networks by Boosting Data Augmentationcited by 0