mlsanityData to 7 Oct 2026

Mitigating Neural Network Overconfidence with Logit Normalization

arXiv 2205.09310 ↗First posted 19 May 2022cs.LGcited by 251

Not on the map. Why?

Abstract (arXiv)

Detecting out-of-distribution inputs is critical for safe deployment of machine learning models in the real world. However, neural networks are known to suffer from the overconfidence issue, where they produce abnormally high confidence for both in- and out-of-distribution inputs. In this work, we show that this issue can be mitigated through Logit Normalization (LogitNorm) -- a simple fix to the cross-entropy loss -- by enforcing a constant vector norm on the logits in training. Our method is motivated by the analysis that the norm of the logit keeps increasing during training, leading to overconfident output. Our key idea behind LogitNorm is thus to decouple the influence of output's norm during network optimization. Trained with LogitNorm, neural networks produce highly distinguishable confidence scores between in- and out-of-distribution data. Extensive experiments demonstrate the superiority of LogitNorm, reducing the average FPR95 by up to 42.30% on common benchmarks.

Key facts. Cited by 251 papers in the map (data through 7 Oct 2026). Builds on 48 of its 70 references that are in the map. First posted on arXiv 19 May 2022.

Builds on

48 of 70 references.

References · 48

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  4. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  5. 2016Densely Connected Convolutional Networkscited by 6,691
  6. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  7. 2017On Calibration of Modern Neural Networkscited by 4,860
  8. 2016Wide Residual Networkscited by 3,057
  9. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,362
  10. 2013Describing Textures in the Wildcited by 2,110
  11. 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,838
  12. 2018A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attackscited by 1,472
36 more
  1. 2015LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loopcited by 1,350
  2. 2017Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networkscited by 1,325
  3. 2020Energy-based Out-of-distribution Detectioncited by 1,296
  4. 2019When Does Label Smoothing Help?cited by 985
  5. 2018CosFace: Large Margin Cosine Loss for Deep Face Recognitioncited by 968
  6. 2018Deep Anomaly Detection with Outlier Exposurecited by 939
  7. 2014Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Imagescited by 774
  8. 2017SphereFace: Deep Hypersphere Embedding for Face Recognitioncited by 733
  9. 2022Out-of-Distribution Detection with Deep Nearest Neighborscited by 596
  10. 2018Predictive Uncertainty Estimation via Prior Networkscited by 555
  11. 2015Towards Open Set Deep Networkscited by 533
  12. 2021ReAct: Out-of-distribution Detection With Rectified Activationscited by 486
  13. 2020CSI: Novelty Detection via Contrastive Learning on Distributionally Shifted Instancescited by 441
  14. 2020Generalized ODIN: Detecting Out-of-distribution Image without Learning from Out-of-distribution Datacited by 408
  15. 2017Training Confidence-calibrated Classifiers for Detecting Out-of-Distribution Samplescited by 389
  16. 2020Calibrating Deep Neural Networks using Focal Losscited by 353
  17. 2021On the Importance of Gradients for Detecting Distributional Shifts in the Wildcited by 338
  18. 2019SelectiveNet: A Deep Neural Network with an Integrated Reject Optioncited by 333
  19. 2022VOS: Learning What You Don't Know by Virtual Outlier Synthesiscited by 300
  20. 2021SSD: A Unified Framework for Self-Supervised Outlier Detectioncited by 264
  21. 2017NormFace: L2 Hypersphere Embedding for Face Verificationcited by 237
  22. 2019Understanding and Improving Layer Normalizationcited by 213
  23. 2015TurkerGaze: Crowdsourcing Saliency with Webcam based Eye Trackingcited by 165
  24. 2020ATOM: Robustifying Out-of-distribution Detection Using Outlier Miningcited by 116
  25. 2021Can multi-label classification networks know what they don't know?cited by 100
  26. 2017L2-constrained Softmax Loss for Discriminative Face Verificationcited by 96
  27. 2022Training OOD Detectors in their Natural Habitatscited by 86
  28. 2019AdaCos: Adaptively Scaling Cosine Logits for Effectively Learning Deep Face Representationscited by 83
  29. 2021Provable Guarantees for Understanding Out-of-distribution Detectioncited by 78
  30. 2020Why Do Better Loss Functions Lead to Less Transferable Features?cited by 73
  31. 2022Unknown-Aware Object Detection: Learning What You Don't Know from Videos in the Wildcited by 70
  32. 2021Open-set Label Noise Can Improve Robustness Against Inherent Label Noisecited by 58
  33. 2021Detecting Corrupted Labels Without Training a Model to Predictcited by 57
  34. 2018Discriminative out-of-distribution detection for semantic segmentationcited by 47
  35. 2021Top-label calibration and multiclass-to-binary reductionscited by 46
  36. 2019Hyperparameter-Free Out-of-Distribution Detection Using Softmax of Scaled Cosine Similaritycited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 12; 2023: 69; 2024: 76; 2025: 60; 2026: 34.

251 papers.

Sort:

By area:

  1. 2022OpenOOD: Benchmarking Generalized Out-of-Distribution Detectioncited by 267
  2. 2023SLCA: Slow Learner with Classifier Alignment for Continual Learning on a Pre-trained Modelcited by 178
  3. 2023OpenOOD v1.5: Enhanced Benchmark for Out-of-Distribution Detectioncited by 178
  4. 2023Non-Parametric Outlier Synthesiscited by 129
  5. 2022POEM: Out-of-Distribution Detection with Posterior Samplingcited by 108
  6. 2024ARGS: Alignment as Reward-Guided Searchcited by 99
  7. 2024Process Reward Model with Q-Value Rankingscited by 78
  8. 2023Dream the Impossible: Outlier Imagination with Diffusion Modelscited by 77
  9. 2023Scaling for Training Time and Post-hoc Out-of-distribution Detection Enhancementcited by 77
  10. 2024Negative Label Guided OOD Detection with Pretrained Vision-Language Modelscited by 66
  11. 2023Not All Neuro-Symbolic Concepts Are Created Equal: Analysis and Mitigation of Reasoning Shortcutscited by 63
  12. 2023Fast Decision Boundary based Out-of-Distribution Detectorcited by 48
38 more
  1. 2022Harnessing Out-Of-Distribution Examples via Augmenting Content and Stylecited by 43
  2. 2024Learning with Mixture of Prototypes for Out-of-Distribution Detectioncited by 43
  3. 2022Mitigating Memorization of Noisy Labels by Clipping the Model Predictioncited by 40
  4. 2023OpenMix: Exploring Outlier Samples for Misclassification Detectioncited by 38
  5. 2023Diversified Outlier Exposure for Out-of-Distribution Detection via Informative Extrapolationcited by 38
  6. 2023Learning to Augment Distributions for Out-of-Distribution Detectioncited by 38
  7. 2024How Does Unlabeled Data Provably Help Out-of-Distribution Detection?cited by 36
  8. 2024BEARS Make Neuro-Symbolic Models Aware of their Reasoning Shortcutscited by 36
  9. 2024Outlier detection by ensembling uncertainty with negative objectnesscited by 34
  10. 2023Detecting Out-of-Distribution Through the Lens of Neural Collapsecited by 33
  11. 2023Out-of-distribution Detection Learning with Unreliable Out-of-distribution Sourcescited by 31
  12. 2024COME: Test-time adaption by Conservatively Minimizing Entropycited by 31
  13. 2022Block Selection Method for Using Feature Norm in Out-of-distribution Detectioncited by 30
  14. 2025Mahalanobis++: Improving OOD Detection via Feature Normalizationcited by 30
  15. 2023Robust Generalization against Photon-Limited Corruptions via Worst-Case Sharpness Minimizationcited by 29
  16. 2023Is Fine-tuning Needed? Pre-trained Language Models Are Near Perfect for Out-of-Domain Detectioncited by 29
  17. 2023gSASRec: Reducing Overconfidence in Sequential Recommendation Trained with Negative Samplingcited by 29
  18. 2024MultiOOD: Scaling Out-of-Distribution Detection for Multiple Modalitiescited by 28
  19. 2024Robust Calibration of Large Vision-Language Adapterscited by 28
  20. 2025Estimating LLM Uncertainty with Evidencecited by 26
  21. 2022Open-Sampling: Exploring Out-of-Distribution data for Re-balancing Long-tailed datasetscited by 25
  22. 2024LEAD: Learning Decomposition for Source-free Universal Domain Adaptationcited by 25
  23. 2024Out-of-Distribution Detection: A Task-Oriented Survey of Recent Advancescited by 24
  24. 2023Out-of-Distribution Detection in Long-Tailed Recognition with Calibrated Outlier Class Learningcited by 23
  25. 2024ConjNorm: Tractable Density Estimation for Out-of-Distribution Detectioncited by 22
  26. 2024A Geometric Explanation of the Likelihood OOD Detection Paradoxcited by 22
  27. 2024LAPT: Label-driven Automated Prompt Tuning for OOD Detection with Vision-Language Modelscited by 21
  28. 2024Robust Semi-supervised Learning by Wisely Leveraging Open-set Datacited by 20
  29. 2024Can We Trust LLMs? Mitigate Overconfidence Bias in LLMs through Knowledge Transfercited by 19
  30. 2025ProtoGCD: Unified and Unbiased Prototype Learning for Generalized Category Discoverycited by 19
  31. 2023From Global to Local: Multi-scale Out-of-distribution Detectioncited by 18
  32. 2024Conformal Prediction via Regression-as-Classificationcited by 18
  33. 2022Weak Proxies are Sufficient and Preferable for Fairness with Missing Sensitive Attributescited by 17
  34. 2023Class Incremental Learning via Likelihood Ratio Based Task Predictioncited by 17
  35. 2024Predictive Dynamic Fusioncited by 17
  36. 2023DOS: Diverse Outlier Sampling for Out-of-Distribution Detectioncited by 16
  37. 2023Unleashing Mask: Explore the Intrinsic Out-of-Distribution Detection Capabilitycited by 16
  38. 2024"Seeing the Big through the Small": Can LLMs Approximate Human Judgment Distributions on NLI from a Few Explanations?cited by 16

50 most cited of 251 All 251 (JSON)