mlsanityData to 7 Oct 2026

Identity Curvature Laplace Approximation for Improved Out-of-Distribution Detection

arXiv 2312.10464 ↗First posted 16 Dec 2023cs.LG cs.AI cs.CVcited by 4

Not on the map. Why?

Abstract (arXiv)

Uncertainty estimation is crucial in safety-critical applications, where robust out-of-distribution (OOD) detection is essential. Traditional Bayesian methods, though effective, are often hindered by high computational demands. As an alternative, Laplace approximation offers a more practical and efficient approach to uncertainty estimation. In this paper, we introduce the Identity Curvature Laplace Approximation (ICLA), a novel method that challenges the conventional posterior covariance formulation by using identity curvature and optimizing prior precision. This innovative design significantly enhances OOD detection performance on well-known datasets such as CIFAR-10, CIFAR-100, and ImageNet, while maintaining calibration scores. We attribute this improvement to the alignment issues between typical feature embeddings and curvature as measured by the Fisher information matrix. Our findings are further supported by demonstrating that incorporating Fisher penalty or sharpness-aware minimization techniques can greatly enhance the uncertainty estimation capabilities of standard Laplace approximation.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 33 of its 50 references that are in the map. First posted on arXiv 16 Dec 2023.

Builds on

33 of 50 references.

References · 33

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  5. 2017On Calibration of Modern Neural Networkscited by 4,860
  6. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  7. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  8. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,362
  9. 2016On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minimacited by 1,337
  10. 2017Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networkscited by 1,325
  11. 2018Deep Anomaly Detection with Outlier Exposurecited by 939
  12. 2015Optimizing Neural Networks with Kronecker-factored Approximate Curvaturecited by 776
21 more
  1. 2014Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Imagescited by 774
  2. 2017Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Datacited by 527
  3. 2021ReAct: Out-of-distribution Detection With Rectified Activationscited by 486
  4. 2020Bayesian Deep Learning and a Probabilistic Perspective of Generalizationcited by 480
  5. 2022ViM: Out-Of-Distribution with Virtual-logit Matchingcited by 404
  6. 2021Revisiting the Calibration of Modern Neural Networkscited by 387
  7. 2021Laplace Redux -- Effortless Bayesian Deep Learningcited by 359
  8. 2020Hands-on Bayesian Neural Networks -- a Tutorial for Deep Learning Userscited by 336
  9. 2021ASAM: Adaptive Sharpness-Aware Minimization for Scale-Invariant Learning of Deep Neural Networkscited by 293
  10. 2015Scalable Bayesian Optimization Using Deep Neural Networkscited by 284
  11. 2017Empirical Analysis of the Hessian of Over-Parametrized Neural Networkscited by 283
  12. 2020Being Bayesian, Even Just a Bit, Fixes Overconfidence in ReLU Networkscited by 227
  13. 2022Extremely Simple Activation Shaping for Out-of-Distribution Detectioncited by 212
  14. 2018Online Structured Laplace Approximations For Overcoming Catastrophic Forgettingcited by 197
  15. 2023OpenOOD v1.5: Enhanced Benchmark for Out-of-Distribution Detectioncited by 178
  16. 2021Scalable Marginal Likelihood Estimation for Model Selection in Deep Learningcited by 105
  17. 2018Universal Statistics of Fisher Information in Deep Neural Networks: Mean Field Approachcited by 89
  18. 2020Catastrophic Fisher Explosion: Early Phase Fisher Matrix Impacts Generalizationcited by 61
  19. 2023Effectively Modeling Time Series with Simple Discrete State Spacescited by 54
  20. 2023NECO: NEural Collapse Based Out-of-distribution detectioncited by 41
  21. 2019Diametrical Risk Minimization: Theory and Computationscited by 15

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 3.

4 papers.

  1. 2026An Isotropic Approach to Efficient Uncertainty Quantification with Gradient Normscited by 1
  2. 2025Uncertainty-driven Embedding Convolutioncited by 0
  3. 2026Monte Carlo Stochastic Depth for Uncertainty Estimation in Deep Learningcited by 0
  4. 2026Uncertainty Estimation for Molecular Diffusion Modelscited by 0