mlsanityData to 7 Oct 2026

Simplifying Neural Network Training Under Class Imbalance

arXiv 2312.02517 ↗First posted 5 Dec 2023cs.LG cs.AIcited by 18

Not on the map. Why?

Abstract (arXiv)

Real-world datasets are often highly class-imbalanced, which can adversely impact the performance of deep learning models. The majority of research on training neural networks under class imbalance has focused on specialized loss functions, sampling techniques, or two-stage training procedures. Notably, we demonstrate that simply tuning existing components of standard deep learning pipelines, such as the batch size, data augmentation, optimizer, and label smoothing, can achieve state-of-the-art performance without any such specialized class imbalance methods. We also provide key prescriptions and considerations for training under class imbalance, and an understanding of why imbalance methods succeed or fail.

Key facts. Cited by 18 papers in the map (data through 7 Oct 2026). Builds on 57 of its 77 references that are in the map. First posted on arXiv 5 Dec 2023.

Builds on

57 of 77 references.

References · 57

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  4. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  5. 2016Densely Connected Convolutional Networkscited by 6,691
  6. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  7. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  8. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,120
  9. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
  10. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  11. 2014Going Deeper with Convolutionscited by 4,231
  12. 2022A ConvNet for the 2020scited by 3,545
38 more
  1. 2016Wide Residual Networkscited by 3,057
  2. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,875
  3. 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,540
  4. 2019Optuna: A Next-generation Hyperparameter Optimization Frameworkcited by 2,455
  5. 2017EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classificationcited by 1,488
  6. 2020Sharpness-Aware Minimization for Efficiently Improving Generalizationcited by 1,404
  7. 2016On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minimacited by 1,337
  8. 2019Class-Balanced Loss Based on Effective Number of Samplescited by 1,239
  9. 2021Swin Transformer V2: Scaling Up Capacity and Resolutioncited by 1,215
  10. 2021VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learningcited by 1,013
  11. 2019Learning Imbalanced Datasets with Label-Distribution-Aware Margin Losscited by 987
  12. 2019When Does Label Smoothing Help?cited by 985
  13. 2019AugMix: A Simple Data Processing Method to Improve Robustness and Uncertaintycited by 897
  14. 2019Decoupling Representation and Classifier for Long-Tailed Recognitioncited by 806
  15. 2018AutoAugment: Learning Augmentation Policies from Datacited by 728
  16. 2021Revisiting Deep Learning Models for Tabular Datacited by 725
  17. 2020Prevalence of Neural Collapse during the terminal phase of deep learning trainingcited by 713
  18. 2019Large-Scale Long-Tailed Recognition in an Open Worldcited by 704
  19. 2021Tabular Data: Deep Learning is Not All You Needcited by 530
  20. 2017A systematic study of the class imbalance problem in convolutional neural networkscited by 510
  21. 2021ResNet strikes back: An improved training procedure in timmcited by 365
  22. 2021SAINT: Improved Neural Networks for Tabular Data via Row Attention and Contrastive Pre-Trainingcited by 310
  23. 2021TrivialAugment: Tuning-free Yet State-of-the-Art Data Augmentationcited by 225
  24. 2023When Do Neural Nets Outperform Boosted Trees on Tabular Data?cited by 218
  25. 2020Rethinking the Value of Labels for Improving Class-Imbalanced Learningcited by 216
  26. 2021Improving Calibration for Long-Tailed Recognitioncited by 212
  27. 2022On Embeddings for Numerical Features in Tabular Deep Learningcited by 185
  28. 2020M2m: Imbalanced Classification via Major-to-minor Translationcited by 134
  29. 2021Self-supervised Learning is More Robust to Dataset Imbalancecited by 122
  30. 2022Neural Collapse: A Review on Modelling Principles and Generalizationcited by 95
  31. 2018Deep Imbalanced Learning for Face Recognition and Attribute Predictioncited by 90
  32. 2013Data-Driven Robust Optimizationcited by 84
  33. 2022PAC-Bayes Compression Bounds So Tight That They Can Explain Generalizationcited by 75
  34. 2017Deep Over-sampling Framework for Classifying Imbalanced Datacited by 58
  35. 2022Can Neural Nets Learn the Same Model Twice? Investigating Reproducibility and Double Descent from the Decision Boundary Perspectivecited by 57
  36. 2019Understanding Generalization through Visualizationscited by 54
  37. 2020Learning to Segment the Tailcited by 54
  38. 2016Factors in Finetuning Deep Model for object detectioncited by 53

50 most cited of 57 All 57 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 12; 2025: 3; 2026: 3.

18 papers.

Sort:

By area:

  1. 2024Fish-Vista: A Multi-Purpose Dataset for Understanding & Identification of Traits from Imagescited by 9
  2. 2025LAMDA: A Longitudinal Android Malware Benchmark for Concept Drift Analysiscited by 6
  3. 2024The Group Robustness is in the Details: Revisiting Finetuning under Spurious Correlationscited by 5
  4. 2024SELECT: A Large-Scale Benchmark of Data Curation Strategies for Image Classificationcited by 5
  5. 2025On the Unreasonable Effectiveness of Last-layer Retrainingcited by 3
  6. 2024Tree species classification at the pixel-level using deep learning and multispectral time series in an imbalanced contextcited by 2
  7. 2024SGW-based Multi-Task Learning in Vision Taskscited by 2
  8. 2024Ensemble Methods for Sequence Classification with Hidden Markov Modelscited by 1
  9. 2024DiffuPT: Class Imbalance Mitigation for Glaucoma Detection via Diffusion Based Generation and Model Pretrainingcited by 1
  10. 2024Leveraging Group Classification with Descending Soft Labeling for Deep Imbalanced Regressioncited by 1
  11. 2025Does Prior Data Matter? Exploring Joint Training in the Context of Few-Shot Class-Incremental Learningcited by 1
  12. 2024Optimizing for ROC Curves on Class-Imbalanced Data by Training over a Family of Loss Functionscited by 0
6 more
  1. 2024Training Over a Distribution of Hyperparameters for Enhanced Performance and Adaptability on Imbalanced Classificationcited by 0
  2. 2024Ratio law: mathematical descriptions for a universal relationship between AI performance and input samplescited by 0
  3. 2024Self-Supervised Learning with Probabilistic Density Labeling for Rainfall Probability Estimationcited by 0
  4. 2026Disentangling Sampling from Training Budget in Class-Imbalanced CT Body Composition Segmentationcited by 0
  5. 2026An interpretable and trustworthy AI framework for large-scale longitudinal structure-pain association studies using data from the Osteoarthritis Initiative (OAI)cited by 0
  6. 2026BP-TTA: Balanced and Prototype-Guided Test-Time Adaptation in Dynamic Scenarioscited by 0