When resampling/reweighting improves feature learning in imbalanced classification?: A toy-model study
Not on the map. Why?
Abstract (arXiv)
A toy model of binary classification is studied with the aim of clarifying the class-wise resampling/reweighting effect on the feature learning performance under the presence of class imbalance. In the analysis, a high-dimensional limit of the input space is taken while keeping the ratio of the dataset size against the input dimension finite and the non-rigorous replica method from statistical mechanics is employed. The result shows that there exists a case in which the no resampling/reweighting situation gives the best feature learning performance irrespectively of the choice of losses or classifiers, supporting recent findings in Cao et al. (2019); Kang et al. (2019). It is also revealed that the key of the result is the symmetry of the loss and the problem setting. Inspired by this, we propose a further simplified model exhibiting the same property in the multiclass setting. These clarify when the class-wise resampling/reweighting becomes effective in imbalanced classification.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 21 of its 54 references that are in the map. First posted on arXiv 9 Sep 2024.
Builds on
21 of 54 references.
References · 21
- 2019Class-Balanced Loss Based on Effective Number of Samplescited by 1,245
- 2019Learning Imbalanced Datasets with Label-Distribution-Aware Margin Losscited by 989
- 2019Decoupling Representation and Classifier for Long-Tailed Recognitioncited by 809
- 2020Prevalence of Neural Collapse during the terminal phase of deep learning trainingcited by 719
- 2019Large-Scale Long-Tailed Recognition in an Open Worldcited by 705
- 2019Self-supervised Visual Feature Learning with Deep Neural Networks: A Surveycited by 657
- 2020Long-tail learning via logit adjustmentcited by 618
- 2020Equalization Loss for Long-Tailed Object Recognitioncited by 285
- 2020Rethinking the Value of Labels for Improving Class-Imbalanced Learningcited by 216
- 2021Exploring Deep Neural Networks via Layer-Peeled Model: Minority Collapse in Imbalanced Trainingcited by 166
- 2020Rethinking Class-Balanced Methods for Long-Tailed Visual Recognition from a Domain Adaptation Perspectivecited by 143
- 2020Finite-sample Analysis of Interpolating Linear Classifiers in the Overparameterized Regimecited by 81
9 more
- 2019Striking the Right Balance with Uncertaintycited by 79
- 2021Label-Imbalanced and Group-Sensitive Classification under Overparameterizationcited by 74
- 2020The role of regularization in classification of high-dimensional noisy Gaussian mixturecited by 63
- 2021Learning Gaussian Mixtures with Generalised Linear Models: Precise Asymptotics in High-dimensionscited by 54
- 2022A Theoretical Analysis of the Learning Dynamics under Class Imbalancecited by 20
- 2022Bias-inducing geometries: an exactly solvable data model with fairness implicationscited by 9
- 2022The Role of Pseudo-labels in Self-training Linear Classifiers on High-dimensional Gaussian Mixture Datacited by 7
- 2024Restoring balance: principled under/oversampling of data for optimal classificationcited by 6
- 2024A replica analysis of under-baggingcited by 4
Built on by
Citing papers per year: 2026: 1.
1 paper.