mlsanityData to 7 Oct 2026

Exploring Weight Balancing on Long-Tailed Recognition Problem

arXiv 2305.16573 ↗First posted 26 May 2023cs.LGcited by 7

Not on the map. Why?

Abstract (arXiv)

Recognition problems in long-tailed data, in which the sample size per class is heavily skewed, have gained importance because the distribution of the sample size per class in a dataset is generally exponential unless the sample size is intentionally adjusted. Various methods have been devised to address these problems.Recently, weight balancing, which combines well-known classical regularization techniques with two-stage training, has been proposed. Despite its simplicity, it is known for its high performance compared with existing methods devised in various ways. However, there is a lack of understanding as to why this method is effective for long-tailed data. In this study, we analyze weight balancing by focusing on neural collapse and the cone effect at each training stage and found that it can be decomposed into an increase in Fisher's discriminant ratio of the feature extractor caused by weight decay and cross entropy loss and implicit logit adjustment caused by weight decay and class-balanced loss. Our analysis enables the training method to be further simplified by reducing the number of training stages to one while increasing accuracy. Code is available at https://github.com/HN410/Exploring-Weight-Balancing-on-Long-Tailed-Recognition-Problem.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 45 of its 64 references that are in the map. First posted on arXiv 26 May 2023.

Builds on

45 of 64 references.

References · 45

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  5. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  6. 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,540
  7. 2016Matching Networks for One Shot Learningcited by 2,246
  8. 2019Class-Balanced Loss Based on Effective Number of Samplescited by 1,239
  9. 2017Visualizing the Loss Landscape of Neural Netscited by 1,126
  10. 2019Learning Imbalanced Datasets with Label-Distribution-Aware Margin Losscited by 987
  11. 2019Decoupling Representation and Classifier for Long-Tailed Recognitioncited by 806
  12. 2020Prevalence of Neural Collapse during the terminal phase of deep learning trainingcited by 713
33 more
  1. 2019Large-Scale Long-Tailed Recognition in an Open Worldcited by 704
  2. 2018An Empirical Study of Example Forgetting during Deep Neural Network Learningcited by 669
  3. 2020Long-tail learning via logit adjustmentcited by 605
  4. 2022Mind the Gap: Understanding the Modality Gap in Multi-modal Contrastive Representation Learningcited by 553
  5. 2021Deep Long-Tailed Learning: A Surveycited by 416
  6. 2017WebVision Database: Visual Learning and Understanding from Web Datacited by 262
  7. 2021Exploring Deep Neural Networks via Layer-Peeled Model: Minority Collapse in Imbalanced Trainingcited by 166
  8. 2021Neural Collapse Under MSE Loss: Proximity to and Dynamics on the Central Pathcited by 156
  9. 2021Well-tuned Simple Nets Excel on Tabular Datasetscited by 151
  10. 2020Feature Space Augmentation for Long-Tailed Datacited by 136
  11. 2021Targeted Supervised Contrastive Learning for Long-Tailed Recognitioncited by 132
  12. 2019An Exponential Learning Rate Schedule for Deep Learningcited by 121
  13. 2020Deep Representation Learning on Long-tailed Data: A Learnable Embedding Augmentation Perspectivecited by 120
  14. 2018Three Mechanisms of Weight Decay Regularizationcited by 112
  15. 2022Long-Tailed Recognition via Weight Balancingcited by 107
  16. 2022Retrieval Augmented Classification for Long-Tail Visual Recognitioncited by 106
  17. 2022Inducing Neural Collapse in Imbalanced Learning: Do We Really Need a Learnable Classifier at the End of Deep Neural Network?cited by 100
  18. 2021An Unconstrained Layer-Peeled Perspective on Neural Collapsecited by 87
  19. 2022Understanding the Generalization Benefit of Normalization Layers: Sharpness Reductioncited by 86
  20. 2022Imbalance Trouble: Revisiting Neural-Collapse Geometrycited by 80
  21. 2021VL-LTR: Learning Class-wise Visual-Linguistic Representation for Long-Tailed Visual Recognitioncited by 65
  22. 2021RSG: A Simple but Effective Module for Learning Imbalanced Datasetscited by 61
  23. 2019Time Matters in Regularizing Deep Networks: Weight Decay and Data Augmentation Affect Early Learning Dynamics, Matter Little Near Convergencecited by 54
  24. 2019Adjusting Decision Boundary for Class Imbalanced Learningcited by 52
  25. 2020Neural Collapse with Cross-Entropy Losscited by 52
  26. 2021A Simple Long-Tailed Recognition Baseline via Vision-Language Modelcited by 49
  27. 2019Four Things Everyone Should Know to Improve Batch Normalizationcited by 28
  28. 2023Inducing Neural Collapse in Deep Long-tailed Learningcited by 28
  29. 2022Delving into Semantic Scale Imbalancecited by 22
  30. 2021On the Periodic Behavior of Neural Network Training with Batch Normalization and Weight Decaycited by 20
  31. 2018On Implicit Filter Level Sparsity in Convolutional Neural Networkscited by 9
  32. 2021Feature Generation for Long-tail Classificationcited by 9
  33. 2022Guidelines for the Regularization of Gammas in Batch Normalization for Deep Residual Networkscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 4; 2025: 0; 2026: 2.

7 papers.

  1. 2024A Systematic Review on Long-Tailed Learningcited by 27
  2. 2024Enhanced Long-Tailed Recognition with Contrastive CutMix Augmentationcited by 6
  3. 2024Rethinking Classifier Re-Training in Long-Tailed Recognition: A Simple Logits Retargeting Approachcited by 2
  4. 2024Multiplicative Logit Adjustment Approximates Neural-Collapse-Aware Decision Boundary Adjustmentcited by 1
  5. 2023On The Relationship Between Continual Learning and Long-Tailed Recognitioncited by 0
  6. 2026Why Not Hyperparameter-Friendly Optimisation? A Monotonic Adaptive Norm Rescaling Approach For Long-Tailed Recognitioncited by 0
  7. 2026Gen2Balance: Generative Balancing for Long-Tailed Video Action Recognitioncited by 0