mlsanityData to 14 Oct 2026

Controlling Neural Collapse Enhances Out-of-Distribution Detection and Transfer Learning

arXiv 2502.10691 ↗First posted 15 Feb 2025cs.LGcited by 11

Not on the map. Why?

Abstract (arXiv)

Out-of-distribution (OOD) detection and OOD generalization are widely studied in Deep Neural Networks (DNNs), yet their relationship remains poorly understood. We empirically show that the degree of Neural Collapse (NC) in a network layer is inversely related with these objectives: stronger NC improves OOD detection but degrades generalization, while weaker NC enhances generalization at the cost of detection. This trade-off suggests that a single feature space cannot simultaneously achieve both tasks. To address this, we develop a theoretical framework linking NC to OOD detection and generalization. We show that entropy regularization mitigates NC to improve generalization, while a fixed Simplex Equiangular Tight Frame (ETF) projector enforces NC for better detection. Based on these insights, we propose a method to control NC at different DNN layers. In experiments, our method excels at both tasks across OOD datasets and DNN architectures. Code for our experiments is available at: https://yousuf907.github.io/ncoodg

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Builds on 71 of its 96 references that are in the map. First posted on arXiv 15 Feb 2025.

Builds on

71 of 96 references.

References · 71

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  4. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  5. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
  6. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,385
  7. 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,850
  8. 2013Fine-Grained Visual Classification of Aircraftcited by 1,726
  9. 2018Group Normalizationcited by 1,524
  10. 2018A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attackscited by 1,485
  11. 2016Understanding intermediate layers using linear classifier probescited by 1,353
  12. 2017Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networkscited by 1,328
38 more
  1. 2019Contrastive Multiview Codingcited by 1,301
  2. 2020Energy-based Out-of-distribution Detectioncited by 1,299
  3. 2018Deep Anomaly Detection with Outlier Exposurecited by 942
  4. 2019Big Transfer (BiT): General Visual Representation Learningcited by 769
  5. 2017Learning to Generalize: Meta-Learning for Domain Generalizationcited by 763
  6. 2020Out-of-Distribution Generalization via Risk Extrapolation (REx)cited by 742
  7. 2020Prevalence of Neural Collapse during the terminal phase of deep learning trainingcited by 719
  8. 2021Do Vision Transformers See Like Convolutional Neural Networks?cited by 675
  9. 2022DualPrompt: Complementary Prompting for Rehearsal-free Continual Learningcited by 642
  10. 2022Out-of-Distribution Detection with Deep Nearest Neighborscited by 600
  11. 2021ReAct: Out-of-distribution Detection With Rectified Activationscited by 486
  12. 2021SWAD: Domain Generalization by Seeking Flat Minimacited by 434
  13. 2021On the Importance of Gradients for Detecting Distributional Shifts in the Wildcited by 339
  14. 2020Learning to Generate Novel Domains for Domain Generalizationcited by 296
  15. 2021Gradient Matching for Domain Generalizationcited by 258
  16. 2019Episodic Training for Domain Generalizationcited by 245
  17. 2021A Geometric Analysis of Neural Collapse with Unconstrained Featurescited by 227
  18. 2021Invariance Principle Meets Information Bottleneck for Out-of-Distribution Generalizationcited by 220
  19. 2021Neural Collapse Under MSE Loss: Proximity to and Dynamics on the Central Pathcited by 158
  20. 2021A Unified Survey on Anomaly, Novelty, Open-Set, and Out-of-Distribution Detection: Solutions and Future Challengescited by 154
  21. 2020Evaluation of Neural Architectures Trained with Square Loss vs Cross-Entropy in Classification Taskscited by 138
  22. 2023In or Out? Fixing ImageNet Out-of-Distribution Detection Evaluationcited by 134
  23. 2018Memory Efficient Experience Replay for Streaming Learningcited by 128
  24. 2022POEM: Out-of-Distribution Detection with Posterior Samplingcited by 108
  25. 2022Better plain ViT baselines for ImageNet-1kcited by 98
  26. 2022Neural Collapse: A Review on Modelling Principles and Generalizationcited by 95
  27. 2021Towards Principled Disentanglement for Domain Generalizationcited by 91
  28. 2021Exploiting Domain-Specific Features to Enhance Domain Generalizationcited by 86
  29. 2022Training OOD Detectors in their Natural Habitatscited by 86
  30. 2022Are All Losses Created Equal: A Neural Collapse Perspectivecited by 78
  31. 2020Why Do Better Loss Functions Lead to Less Transferable Features?cited by 73
  32. 2020Permuted AdaIN: Reducing the Bias Towards Global Statistics in Image Classificationcited by 63
  33. 2023Feed Two Birds with One Scone: Exploiting Wild Data for Both Out-of-Distribution Generalization and Detectioncited by 57
  34. 2022Boosting Out-of-distribution Detection with Typical Featurescited by 48
  35. 2021Revisiting the Transferability of Supervised Pretraining: an MLP Perspectivecited by 36
  36. 2023Does progress on ImageNet transfer to real-world datasets?cited by 35
  37. 2023The Tunnel Effect: Building Data Representations in Deep Neural Networkscited by 34
  38. 2023On the Connection between Pre-training Data Diversity and Fine-tuning Robustnesscited by 25

50 most cited of 71 All 71 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 9.

11 papers.

  1. 2025NEO: No-Optimization Test-Time Adaptation through Latent Re-Centeringcited by 8
  2. 2026Diagnosing Generalization Failures from Representational Geometry Markerscited by 2
  3. 2025A Good Start Matters: Enhancing Continual Learning with Data-Driven Weight Initializationcited by 1
  4. 2026GEPC: Group-Equivariant Posterior Consistency for Out-of-Distribution Detection in Diffusion Modelscited by 1
  5. 2026Unlocking ImageNet's Multi-Object Nature: Automated Large-Scale Multilabel Annotationcited by 0
  6. 2026Prototype Fusion: A Training-Free Multi-Layer Approach to OOD Detectioncited by 0
  7. 2026Cluster-Aware Neural Collapse Prompt Tuning for Long-Tailed Generalization of Vision-Language Modelscited by 0
  8. 2026Neural Collapse by Design: Learning Class Prototypes on the Hyperspherecited by 0
  9. 2026MM++: Post-Hoc Scale-Invariant Multilayer OOD Detection via Top-K Gated Feature Fusioncited by 0
  10. 2026SwinAD: Multi-stage feature reconstruction for unsupervised industrial anomaly detectioncited by 0
  11. 2026TOOD: Task-Aware Out-of-Distribution Score Calibration for Continual Learnerscited by 0