Multiplicative Logit Adjustment Approximates Neural-Collapse-Aware Decision Boundary Adjustment
Not on the map. Why?
Abstract (arXiv)
Real-world data distributions are often highly skewed. This has spurred a growing body of research on long-tailed recognition, aimed at addressing the imbalance in training classification models. Among the methods studied, multiplicative logit adjustment (MLA) stands out as a simple and effective method. What theoretical foundation explains the effectiveness of this heuristic method? We provide a justification for the effectiveness of MLA with the following two-step process. First, we develop a theory that adjusts optimal decision boundaries by estimating feature spread on the basis of neural collapse. Second, we demonstrate that MLA approximates this optimal method. Additionally, through experiments on long-tailed datasets, we illustrate the practical usefulness of MLA under more realistic conditions. We also offer experimental insights to guide the tuning of MLA hyperparameters.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 47 of its 67 references that are in the map. First posted on arXiv 26 Sep 2024.
Builds on
47 of 67 references.
References · 47
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
- 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,540
- 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,250
- 2019Class-Balanced Loss Based on Effective Number of Samplescited by 1,239
- 2019Learning Imbalanced Datasets with Label-Distribution-Aware Margin Losscited by 987
- 2019Decoupling Representation and Classifier for Long-Tailed Recognitioncited by 806
- 2020Prevalence of Neural Collapse during the terminal phase of deep learning trainingcited by 713
- 2019Large-Scale Long-Tailed Recognition in an Open Worldcited by 704
- 2020Long-tail learning via logit adjustmentcited by 605
- 2021Deep Long-Tailed Learning: A Surveycited by 416
35 more
- 2017Size-Independent Sample Complexity of Neural Networkscited by 289
- 2017WebVision Database: Visual Learning and Understanding from Web Datacited by 262
- 2020Long-Tailed Classification by Keeping the Good and Removing the Bad Momentum Causal Effectcited by 256
- 2018What is the Effect of Importance Weighting in Deep Learning?cited by 250
- 2021Exploring Deep Neural Networks via Layer-Peeled Model: Minority Collapse in Imbalanced Trainingcited by 166
- 2021Neural Collapse Under MSE Loss: Proximity to and Dynamics on the Central Pathcited by 156
- 2021Well-tuned Simple Nets Excel on Tabular Datasetscited by 151
- 2020Feature Space Augmentation for Long-Tailed Datacited by 136
- 2021Targeted Supervised Contrastive Learning for Long-Tailed Recognitioncited by 132
- 2020Deep Representation Learning on Long-tailed Data: A Learnable Embedding Augmentation Perspectivecited by 120
- 2022Retrieval Augmented Classification for Long-Tail Visual Recognitioncited by 106
- 2021On the Role of Neural Collapse in Transfer Learningcited by 100
- 2022CODA: A Real-World Road Corner Case Dataset for Object Detection in Autonomous Drivingcited by 100
- 2022Inducing Neural Collapse in Imbalanced Learning: Do We Really Need a Learnable Classifier at the End of Deep Neural Network?cited by 100
- 2021An Unconstrained Layer-Peeled Perspective on Neural Collapsecited by 87
- 2022Imbalance Trouble: Revisiting Neural-Collapse Geometrycited by 80
- 2020Identifying and Compensating for Feature Deviation in Imbalanced Deep Learningcited by 76
- 2022A Survey on Long-Tailed Visual Recognitioncited by 75
- 2023Long-tailed Visual Recognition via Gaussian Clouded Logit Adjustmentcited by 75
- 2021Label-Imbalanced and Group-Sensitive Classification under Overparameterizationcited by 74
- 2021VL-LTR: Learning Class-wise Visual-Linguistic Representation for Long-Tailed Visual Recognitioncited by 65
- 2021RSG: A Simple but Effective Module for Learning Imbalanced Datasetscited by 61
- 2019Improved Sample Complexities for Deep Networks and Robust Classification via an All-Layer Margincited by 56
- 2019Adjusting Decision Boundary for Class Imbalanced Learningcited by 52
- 2020Neural Collapse with Cross-Entropy Losscited by 52
- 2021A Simple Long-Tailed Recognition Baseline via Vision-Language Modelcited by 49
- 2023Are Transformers with One Layer Self-Attention Using Low-Rank Weight Matrices Universal Approximators?cited by 46
- 2022Partial and Asymmetric Contrastive Learning for Out-of-Distribution Detection in Long-Tailed Recognitioncited by 44
- 2022Understanding Why Generalized Reweighting Does Not Improve Over ERMcited by 29
- 2023Inducing Neural Collapse in Deep Long-tailed Learningcited by 28
- 2023A Unified Perspective for Loss-Oriented Imbalanced Learning via Localizationcited by 24
- 2022Delving into Semantic Scale Imbalancecited by 22
- 2024Neural Collapse for Cross-entropy Class-Imbalanced Learning with Unconstrained ReLU Feature Modelcited by 19
- 2023Exploring Weight Balancing on Long-Tailed Recognition Problemcited by 7
- 2022Inverse Image Frequency for Long-tailed Image Recognitioncited by 6
Built on by
Citing papers per year: 2026: 1.
1 paper.