mlsanityData to 7 Oct 2026

Linking Neural Collapse and L2 Normalization with Improved Out-of-Distribution Detection in Deep Neural Networks

arXiv 2209.08378 ↗First posted 17 Sep 2022cs.LGcited by 23

Not on the map. Why?

Abstract (arXiv)

We propose a simple modification to standard ResNet architectures--L2 normalization over feature space--that substantially improves out-of-distribution (OoD) performance on the previously proposed Deep Deterministic Uncertainty (DDU) benchmark. We show that this change also induces early Neural Collapse (NC), an effect linked to better OoD performance. Our method achieves comparable or superior OoD detection scores and classification accuracy in a small fraction of the training time of the benchmark. Additionally, it substantially improves worst case OoD performance over multiple, randomly initialized models. Though we do not suggest that NC is the sole mechanism or a comprehensive explanation for OoD behaviour in deep neural networks (DNN), we believe NC's simple mathematical and geometric structure can provide a framework for analysis of this complex phenomenon in future work.

Key facts. Cited by 23 papers in the map (data through 7 Oct 2026). Builds on 20 of its 27 references that are in the map. First posted on arXiv 17 Sep 2022.

Builds on

20 of 27 references.

References · 20

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  3. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  4. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,362
  5. 2018A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attackscited by 1,472
  6. 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,274
  7. 2017SphereFace: Deep Hypersphere Embedding for Face Recognitioncited by 733
  8. 2020Prevalence of Neural Collapse during the terminal phase of deep learning trainingcited by 713
  9. 2020Generalized ODIN: Detecting Out-of-distribution Image without Learning from Out-of-distribution Datacited by 408
  10. 2020Simple and Principled Uncertainty Estimation with Deterministic Deep Learning via Distance Awarenesscited by 389
  11. 2018Why ReLU networks yield high-confidence predictions far away from the training data and how to mitigate the problemcited by 323
  12. 2018Failing Loudly: An Empirical Study of Methods for Detecting Dataset Shiftcited by 240
8 more
  1. 2021A Geometric Analysis of Neural Collapse with Unconstrained Featurescited by 226
  2. 2020Neural collapse with unconstrained featurescited by 140
  3. 2018Spreading vectors for similarity searchcited by 114
  4. 2021On Feature Collapse and Deep Kernel Learning for Single Forward Pass Uncertaintycited by 87
  5. 2021An Unconstrained Layer-Peeled Perspective on Neural Collapsecited by 87
  6. 2020Robust Out-of-distribution Detection for Neural Networkscited by 57
  7. 2020Neural Collapse with Cross-Entropy Losscited by 52
  8. 2021Can convolutional ResNets approximately preserve input distances? A frequency analysis perspectivecited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 8; 2025: 8; 2026: 4.

23 papers.

Sort:

  1. 2023Deep Neural Collapse Is Provably Optimal for the Deep Unconstrained Features Modelcited by 43
  2. 2023NECO: NEural Collapse Based Out-of-distribution detectioncited by 41
  3. 2024Linguistic Collapse: Neural Collapse in (Large) Language Modelscited by 36
  4. 2025Mahalanobis++: Improving OOD Detection via Feature Normalizationcited by 30
  5. 2024Average gradient outer product as a mechanism for deep neural collapsecited by 21
  6. 2024Neural Collapse versus Low-rank Bias: Is Deep Neural Collapse Really Optimal?cited by 16
  7. 2024Wide Neural Networks Trained with Weight Decay Provably Exhibit Neural Collapsecited by 14
  8. 2024Progressive Feedforward Collapse of ResNet Trainingcited by 12
  9. 2024Pursuing Feature Separation based on Neural Collapse for Out-of-Distribution Detectioncited by 12
  10. 2025Controlling Neural Collapse Enhances Out-of-Distribution Detection and Transfer Learningcited by 11
  11. 2024EPA: Neural Collapse Inspired Robust Out-of-Distribution Detectorcited by 10
  12. 2025Neural Collapse is Globally Optimal in Deep Regularized ResNets and Transformerscited by 9
11 more
  1. 2025Improving Out-of-Distribution Detection via Dynamic Covariance Calibrationcited by 4
  2. 2023Exploring Simple, High Quality Out-of-Distribution Detection with L2 Normalizationcited by 3
  3. 2025Unpacking Softmax: How Temperature Drives Representation Collapse, Compression, and Generalizationcited by 3
  4. 2025BootOOD: Self-Supervised Out-of-Distribution Detection via Synthetic Sample Exposure under Neural Collapsecited by 1
  5. 2025Heads collapse, features stay: Why Replay needs big bufferscited by 1
  6. 2024Double Descent Meets Out-of-Distribution Detection: Theoretical Insights and Empirical Analysis on the role of model complexitycited by 0
  7. 2025Reliable Active Learning from Unreliable Labels via Neural Collapse Geometrycited by 0
  8. 2026Beyond the Class Subspace: Teacher-Guided Training for Reliable Out-of-Distribution Detection in Single-Domain Modelscited by 0
  9. 2026Don't Collapse Your Features: Why CenterLoss Hurts OOD Detection and Multi-Scale Mahalanobis Winscited by 0
  10. 2026The role of class encoding in neural collapsecited by 0
  11. 2026The Loss Is Not Enough: Sampling Conditions and Inductive Bias in Contrastive Representation Learningcited by 0