mlsanityData to 7 Oct 2026

Exploring Simple, High Quality Out-of-Distribution Detection with L2 Normalization

arXiv 2306.04072 ↗First posted 7 Jun 2023cs.LGcited by 3

Not on the map. Why?

Abstract (arXiv)

We demonstrate that L2 normalization over feature space can produce capable performance for Out-of-Distribution (OoD) detection for some models and datasets. Although it does not demonstrate outright state-of-the-art performance, this method is notable for its extreme simplicity: it requires only two addition lines of code, and does not need specialized loss functions, image augmentations, outlier exposure or extra parameter tuning. We also observe that training may be more efficient for some datasets and architectures. Notably, only 60 epochs with ResNet18 on CIFAR10 (or 100 epochs with ResNet50) can produce performance within two percentage points (AUROC) of several state-of-the-art methods for some near and far OoD datasets. We provide theoretical and empirical support for this method, and demonstrate viability across five architectures and three In-Distribution (ID) datasets.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 24 of its 31 references that are in the map. First posted on arXiv 7 Jun 2023.

Builds on

24 of 31 references.

References · 24

  1. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  2. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  3. 2022A ConvNet for the 2020scited by 3,545
  4. 2020Prevalence of Neural Collapse during the terminal phase of deep learning trainingcited by 713
  5. 2019Scaling Out-of-Distribution Detection for Real-World Settingscited by 465
  6. 2021Open-Set Recognition: a Good Closed-Set Classifier is All You Need?cited by 351
  7. 2020A Metric Learning Reality Checkcited by 317
  8. 2021Escaping the Big Data Paradigm with Compact Transformerscited by 261
  9. 2022Mitigating Neural Network Overconfidence with Logit Normalizationcited by 251
  10. 2018Failing Loudly: An Empirical Study of Methods for Detecting Dataset Shiftcited by 240
  11. 2017NormFace: L2 Hypersphere Embedding for Face Verificationcited by 237
  12. 2021A Geometric Analysis of Neural Collapse with Unconstrained Featurescited by 226
12 more
  1. 2020Neural collapse with unconstrained featurescited by 140
  2. 2017L2-constrained Softmax Loss for Discriminative Face Verificationcited by 96
  3. 2021An Unconstrained Layer-Peeled Perspective on Neural Collapsecited by 87
  4. 2021Standardized Max Logits: A Simple yet Effective Approach for Identifying Unexpected Road Obstacles in Urban-Scene Segmentationcited by 77
  5. 2021Dissecting Supervised Contrastive Learningcited by 64
  6. 2020Robust Out-of-distribution Detection for Neural Networkscited by 57
  7. 2020Neural Collapse with Cross-Entropy Losscited by 52
  8. 2022Neural Collapse with Normalized Features: A Geometric Analysis over the Riemannian Manifoldcited by 50
  9. 2020Coherent Gradients: An Approach to Understanding Generalization in Gradient Descent-based Optimizationcited by 41
  10. 2022Block Selection Method for Using Feature Norm in Out-of-distribution Detectioncited by 30
  11. 2022Linking Neural Collapse and L2 Normalization with Improved Out-of-Distribution Detection in Deep Neural Networkscited by 23
  12. 2023T2FNorm: Extremely Simple Scaled Train-time Feature Normalization for OOD Detectioncited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.

  1. 2025Mahalanobis++: Improving OOD Detection via Feature Normalizationcited by 30
  2. 2026The Loss Is Not Enough: Sampling Conditions and Inductive Bias in Contrastive Representation Learningcited by 0
  3. 2026MM++: Post-Hoc Scale-Invariant Multilayer OOD Detection via Top-K Gated Feature Fusioncited by 0