# Simplifying Neural Network Training Under Class Imbalance

arXiv: https://arxiv.org/abs/2312.02517 · HTML: https://mlsanity.com/paper/2312.02517/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 57 of its 77 references that are in the map.
- First posted on arXiv 5 Dec 2023.

## Builds on (most cited first, 30 of 57)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [Wide Residual Networks](https://mlsanity.com/paper/1605.07146/) (2016)
- [CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features](https://mlsanity.com/paper/1905.04899/) (2019)
- [Aggregated Residual Transformations for Deep Neural Networks](https://mlsanity.com/paper/1611.05431/) (2016)
- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classification](https://mlsanity.com/paper/1709.00029/) (2017)
- [Sharpness-Aware Minimization for Efficiently Improving Generalization](https://mlsanity.com/paper/2010.01412/) (2020)
- [On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima](https://mlsanity.com/paper/1609.04836/) (2016)
- [Class-Balanced Loss Based on Effective Number of Samples](https://mlsanity.com/paper/1901.05555/) (2019)
- [Swin Transformer V2: Scaling Up Capacity and Resolution](https://mlsanity.com/paper/2111.09883/) (2021)
- [VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning](https://mlsanity.com/paper/2105.04906/) (2021)
- [Learning Imbalanced Datasets with Label-Distribution-Aware Margin Loss](https://mlsanity.com/paper/1906.07413/) (2019)
- [When Does Label Smoothing Help?](https://mlsanity.com/paper/1906.02629/) (2019)
- [AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty](https://mlsanity.com/paper/1912.02781/) (2019)
- [Decoupling Representation and Classifier for Long-Tailed Recognition](https://mlsanity.com/paper/1910.09217/) (2019)
- [Revisiting Deep Learning Models for Tabular Data](https://mlsanity.com/paper/2106.11959/) (2021)
- [AutoAugment: Learning Augmentation Policies from Data](https://mlsanity.com/paper/1805.09501/) (2018)
- [Prevalence of Neural Collapse during the terminal phase of deep learning training](https://mlsanity.com/paper/2008.08186/) (2020)
- [Large-Scale Long-Tailed Recognition in an Open World](https://mlsanity.com/paper/1904.05160/) (2019)

## Built on by (most cited first, 18 of 18)

- [Fish-Vista: A Multi-Purpose Dataset for Understanding & Identification of Traits from Images](https://mlsanity.com/paper/2407.08027/) (2024)
- [The Group Robustness is in the Details: Revisiting Finetuning under Spurious Correlations](https://mlsanity.com/paper/2407.13957/) (2024)
- [LAMDA: A Longitudinal Android Malware Benchmark for Concept Drift Analysis](https://mlsanity.com/paper/2505.18551/) (2025)
- [SELECT: A Large-Scale Benchmark of Data Curation Strategies for Image Classification](https://mlsanity.com/paper/2410.05057/) (2024)
- [Tree species classification at the pixel-level using deep learning and multispectral time series in an imbalanced context](https://mlsanity.com/paper/2408.08887/) (2024)
- [On the Unreasonable Effectiveness of Last-layer Retraining](https://mlsanity.com/paper/2512.01766/) (2025)
- [SGW-based Multi-Task Learning in Vision Tasks](https://mlsanity.com/paper/2410.03778/) (2024)
- [Ensemble Methods for Sequence Classification with Hidden Markov Models](https://mlsanity.com/paper/2409.07619/) (2024)
- [DiffuPT: Class Imbalance Mitigation for Glaucoma Detection via Diffusion Based Generation and Model Pretraining](https://mlsanity.com/paper/2412.03629/) (2024)
- [Leveraging Group Classification with Descending Soft Labeling for Deep Imbalanced Regression](https://mlsanity.com/paper/2412.12327/) (2024)
- [Does Prior Data Matter? Exploring Joint Training in the Context of Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2503.10003/) (2025)
- [Optimizing for ROC Curves on Class-Imbalanced Data by Training over a Family of Loss Functions](https://mlsanity.com/paper/2402.05400/) (2024)
- [Training Over a Distribution of Hyperparameters for Enhanced Performance and Adaptability on Imbalanced Classification](https://mlsanity.com/paper/2410.03588/) (2024)
- [Ratio law: mathematical descriptions for a universal relationship between AI performance and input samples](https://mlsanity.com/paper/2411.00913/) (2024)
- [Self-Supervised Learning with Probabilistic Density Labeling for Rainfall Probability Estimation](https://mlsanity.com/paper/2412.05825/) (2024)
- [Disentangling Sampling from Training Budget in Class-Imbalanced CT Body Composition Segmentation](https://mlsanity.com/paper/2605.20405/) (2026)
- [An interpretable and trustworthy AI framework for large-scale longitudinal structure-pain association studies using data from the Osteoarthritis Initiative (OAI)](https://mlsanity.com/paper/2606.05357/) (2026)
- [BP-TTA: Balanced and Prototype-Guided Test-Time Adaptation in Dynamic Scenarios](https://mlsanity.com/paper/2606.31420/) (2026)

## Abstract (arXiv)

Real-world datasets are often highly class-imbalanced, which can adversely impact the performance of deep learning models. The majority of research on training neural networks under class imbalance has focused on specialized loss functions, sampling techniques, or two-stage training procedures. Notably, we demonstrate that simply tuning existing components of standard deep learning pipelines, such as the batch size, data augmentation, optimizer, and label smoothing, can achieve state-of-the-art performance without any such specialized class imbalance methods. We also provide key prescriptions and considerations for training under class imbalance, and an understanding of why imbalance methods succeed or fail.