# Don't Collapse Your Features: Why CenterLoss Hurts OOD Detection and Multi-Scale Mahalanobis Wins

arXiv: https://arxiv.org/abs/2605.21493 · HTML: https://mlsanity.com/paper/2605.21493/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 25 of its 40 references that are in the map.
- First posted on arXiv 10 Apr 2026.

## Builds on (most cited first, 25 of 25)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization](https://mlsanity.com/paper/2006.16241/) (2020)
- [A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks](https://mlsanity.com/paper/1807.03888/) (2018)
- [Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks](https://mlsanity.com/paper/1706.02690/) (2017)
- [Energy-based Out-of-distribution Detection](https://mlsanity.com/paper/2010.03759/) (2020)
- [Evidential Deep Learning to Quantify Classification Uncertainty](https://mlsanity.com/paper/1806.01768/) (2018)
- [Out-of-Distribution Detection with Deep Nearest Neighbors](https://mlsanity.com/paper/2204.06507/) (2022)
- [On Mixup Training: Improved Calibration and Predictive Uncertainty for Deep Neural Networks](https://mlsanity.com/paper/1905.11001/) (2019)
- [Epistemic Neural Networks](https://mlsanity.com/paper/2107.08924/) (2021)
- [A Comprehensive Survey of Mixture-of-Experts: Algorithms, Theory, and Applications](https://mlsanity.com/paper/2503.07137/) (2025)
- [A Survey on Uncertainty Quantification Methods for Deep Learning](https://mlsanity.com/paper/2302.13425/) (2023)
- [Robust Out-of-distribution Detection for Neural Networks](https://mlsanity.com/paper/2003.09711/) (2020)
- [Detecting Out-of-Distribution Through the Lens of Neural Collapse](https://mlsanity.com/paper/2311.01479/) (2023)
- [Adaptive Temperature Scaling for Robust Calibration of Deep Neural Networks](https://mlsanity.com/paper/2208.00461/) (2022)
- [Linking Neural Collapse and L2 Normalization with Improved Out-of-Distribution Detection in Deep Neural Networks](https://mlsanity.com/paper/2209.08378/) (2022)
- [Multi-Class Data Description for Out-of-distribution Detection](https://mlsanity.com/paper/2104.00941/) (2021)
- [Self-Calibrating Conformal Prediction](https://mlsanity.com/paper/2402.07307/) (2024)
- [On the Validity of Bayesian Neural Networks for Uncertainty Estimation](https://mlsanity.com/paper/1912.01530/) (2019)
- [Quantification of Deep Neural Network Prediction Uncertainties for VVUQ of Machine Learning Models](https://mlsanity.com/paper/2206.14615/) (2022)
- [Controllable-LPMoE: Adapting to Challenging Object Segmentation via Dynamic Local Priors from Mixture-of-Experts](https://mlsanity.com/paper/2510.21114/) (2025)
- [Performance Analysis of Out-of-Distribution Detection on Trained Neural Networks](https://mlsanity.com/paper/2204.12378/) (2022)
- [Theory of Mixture-of-Experts for Mobile Edge Computing](https://mlsanity.com/paper/2412.15690/) (2024)

## Abstract (arXiv)

The ability to detect out-of-distribution (OOD) inputs is fundamental to safe deployment of machine learning systems. Yet, current methods often rely on feature representations that are optimised solely for classification accuracy, neglecting the distinct requirements of epistemic uncertainty. We introduce GOEN (Geometry-Optimised Epistemic Network), a simple pipeline that combines multi-scale features, L2 normalisation, Mahalanobis distance, and a calibration head trained with real hard OOD examples. Through systematic ablation we uncover a counter-intuitive finding: CenterLoss, a popular regulariser for feature compactness, significantly degrades OOD detection performance, reducing average OOD AUROC from 0.9483 to 0.9366 despite improving classification accuracy. The best variant, GOEN-NoCenterLoss, achieves an average OOD AUROC of 0.9483, surpassing all baselines including deep ensembles (0.8827), KNN (0.8967), and ODIN (0.8870) on CIFAR-10 benchmarks, while maintaining competitive in-distribution accuracy. Our results challenge the prevailing assumption that better classification geometry automatically leads to better epistemic uncertainty. Instead, we show that overly tight feature clusters compress inter-class margins and distort the covariance structure needed for effective OOD detection. GOEN is efficient, training in under 20 minutes on a single GPU, and provides a practical blueprint for building AI systems that reliably recognise their own limitations.