Representation Norm Amplification for Out-of-Distribution Detection in Long-Tail Learning
Not on the map. Why?
Abstract (arXiv)
Detecting out-of-distribution (OOD) samples is a critical task for reliable machine learning. However, it becomes particularly challenging when the models are trained on long-tailed datasets, as the models often struggle to distinguish tail-class in-distribution samples from OOD samples. We examine the main challenges in this problem by identifying the trade-offs between OOD detection and in-distribution (ID) classification, faced by existing methods. We then introduce our method, called textitRepresentation Norm Amplification (RNA), which solves this challenge by decoupling the two problems. The main idea is to use the norm of the representation as a new dimension for OOD detection, and to develop a training method that generates a noticeable discrepancy in the representation norm between ID and OOD data, while not perturbing the feature learning for ID classification. Our experiments show that RNA achieves superior performance in both OOD detection and classification compared to the state-of-the-art methods, by 1.70% and 9.46% in FPR95 and 2.43% and 6.87% in classification accuracy on CIFAR10-LT and ImageNet-LT, respectively. The code for this work is available at https://github.com/dgshin21/RNA.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 57 of its 71 references that are in the map. First posted on arXiv 20 Aug 2024.
Builds on
57 of 71 references.
References · 57
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
- 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
- 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,875
- 2020Supervised Contrastive Learningcited by 2,803
- 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,362
- 2013Describing Textures in the Wildcited by 2,110
- 2018A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attackscited by 1,472
- 2015LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loopcited by 1,350
- 2017Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networkscited by 1,325
- 2020Energy-based Out-of-distribution Detectioncited by 1,296
- 2019Class-Balanced Loss Based on Effective Number of Samplescited by 1,239
38 more
- 2019Learning Imbalanced Datasets with Label-Distribution-Aware Margin Losscited by 987
- 2018Deep Anomaly Detection with Outlier Exposurecited by 939
- 2019Decoupling Representation and Classifier for Long-Tailed Recognitioncited by 806
- 2014Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Imagescited by 774
- 2019Large-Scale Long-Tailed Recognition in an Open Worldcited by 704
- 2020Long-tail learning via logit adjustmentcited by 605
- 2022Out-of-Distribution Detection with Deep Nearest Neighborscited by 596
- 2018Predictive Uncertainty Estimation via Prior Networkscited by 555
- 2015Towards Open Set Deep Networkscited by 533
- 2019Using Self-Supervised Learning Can Improve Model Robustness and Uncertaintycited by 511
- 2021ReAct: Out-of-distribution Detection With Rectified Activationscited by 486
- 2020CSI: Novelty Detection via Contrastive Learning on Distributionally Shifted Instancescited by 441
- 2020Generalized ODIN: Detecting Out-of-distribution Image without Learning from Out-of-distribution Datacited by 408
- 2022ViM: Out-Of-Distribution with Virtual-logit Matchingcited by 404
- 2017Training Confidence-calibrated Classifiers for Detecting Out-of-Distribution Samplescited by 389
- 2017A Downsampled Variant of ImageNet as an Alternative to the CIFAR datasetscited by 374
- 2021Open-Set Recognition: a Good Closed-Set Classifier is All You Need?cited by 351
- 2021On the Importance of Gradients for Detecting Distributional Shifts in the Wildcited by 338
- 2018Why ReLU networks yield high-confidence predictions far away from the training data and how to mitigate the problemcited by 323
- 2020Equalization Loss for Long-Tailed Object Recognitioncited by 285
- 2021SSD: A Unified Framework for Self-Supervised Outlier Detectioncited by 264
- 2022Mitigating Neural Network Overconfidence with Logit Normalizationcited by 251
- 2021Parametric Contrastive Learningcited by 217
- 2022Extremely Simple Activation Shaping for Out-of-Distribution Detectioncited by 212
- 2021DICE: Leveraging Sparsification for Out-of-Distribution Detectioncited by 166
- 2018Reducing Network Agnostophobiacited by 165
- 2022Balanced Contrastive Learning for Long-Tailed Visual Recognitioncited by 143
- 2021Targeted Supervised Contrastive Learning for Long-Tailed Recognitioncited by 132
- 2022How to Exploit Hyperspherical Embeddings for Out-of-Distribution Detection?cited by 128
- 2020ATOM: Robustifying Out-of-distribution Detection Using Outlier Miningcited by 116
- 2022POEM: Out-of-Distribution Detection with Posterior Samplingcited by 108
- 2021Semantically Coherent Out-of-Distribution Detectioncited by 107
- 2021Can multi-label classification networks know what they don't know?cited by 100
- 2022Training OOD Detectors in their Natural Habitatscited by 86
- 2022RankFeat: Rank-1 Feature Removal for Out-of-distribution Detectioncited by 52
- 2022Boosting Out-of-distribution Detection with Typical Featurescited by 48
- 2022Partial and Asymmetric Contrastive Learning for Out-of-Distribution Detection in Long-Tailed Recognitioncited by 44
- 2021Neural Mean Discrepancy for Efficient Out-of-Distribution Detectioncited by 42
50 most cited of 57 All 57 (JSON)
Built on by
Citing papers per year: 2025: 1.
1 paper.
- 2025SEQR: Secure and Efficient QR-based LoRA Routingcited by 1
- 2025SEQR: Secure and Efficient QR-based LoRA Routing22 Sep 2025