# Overfitting of neural nets under class imbalance: Analysis and improvements for segmentation

arXiv: https://arxiv.org/abs/1907.10982 · HTML: https://mlsanity.com/paper/1907.10982/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Jul 2019.

## Built on by (most cited first, 27 of 27)

- [Unified Focal loss: Generalising Dice and cross entropy-based losses to handle class imbalanced medical image segmentation](https://mlsanity.com/paper/2102.04525/) (2021)
- [Conditional Diffusion Models for Semantic 3D Brain MRI Synthesis](https://mlsanity.com/paper/2305.18453/) (2023)
- [Rethinking Semi-Supervised Medical Image Segmentation: A Variance-Reduction Perspective](https://mlsanity.com/paper/2302.01735/) (2023)
- [Analyzing Overfitting under Class Imbalance in Neural Networks for Image Segmentation](https://mlsanity.com/paper/2102.10365/) (2021)
- [Deep learning-based synthetic-CT generation in radiotherapy and PET: a review](https://mlsanity.com/paper/2102.02734/) (2021)
- [Enhancing MR Image Segmentation with Realistic Adversarial Data Augmentation](https://mlsanity.com/paper/2108.03429/) (2021)
- [Constrained Optimization to Train Neural Networks on Critical and Under-Represented Classes](https://mlsanity.com/paper/2102.12894/) (2021)
- [InPL: Pseudo-labeling the Inliers First for Imbalanced Semi-supervised Learning](https://mlsanity.com/paper/2303.07269/) (2023)
- [VisCUIT: Visual Auditor for Bias in CNN Image Classifier](https://mlsanity.com/paper/2204.05899/) (2022)
- [RESECT-SEG: Open access annotations of intra-operative brain tumor ultrasound images](https://mlsanity.com/paper/2207.07494/) (2022)
- [Joint Modeling of Image and Label Statistics for Enhancing Model Generalizability of Medical Image Segmentation](https://mlsanity.com/paper/2206.04336/) (2022)
- [Co-Learning Meets Stitch-Up for Noisy Multi-label Visual Recognition](https://mlsanity.com/paper/2307.00880/) (2023)
- [Detecting, Localising and Classifying Polyps from Colonoscopy Videos using Deep Learning](https://mlsanity.com/paper/2101.03285/) (2021)
- [Balancing Multi-Target Semi-Supervised Medical Image Segmentation with Collaborative Generalist and Specialists](https://mlsanity.com/paper/2504.00862/) (2025)
- [Transesophageal Echocardiography Generation using Anatomical Models](https://mlsanity.com/paper/2410.06781/) (2024)
- [On Discriminative Probabilistic Modeling for Self-Supervised Representation Learning](https://mlsanity.com/paper/2410.09156/) (2024)
- [MetaBalance: High-Performance Neural Networks for Class-Imbalanced Data](https://mlsanity.com/paper/2106.09643/) (2021)
- [Empirical Study of Overfitting in Deep FNN Prediction Models for Breast Cancer Metastasis](https://mlsanity.com/paper/2208.02150/) (2022)
- [Context Label Learning: Improving Background Class Representations in Semantic Segmentation](https://mlsanity.com/paper/2212.08423/) (2022)
- [Ranking Regularization for Critical Rare Classes: Minimizing False Positives at a High True Positive Rate](https://mlsanity.com/paper/2304.00049/) (2023)
- [The Effects of Mixed Sample Data Augmentation are Class Dependent](https://mlsanity.com/paper/2307.09136/) (2023)
- [Dr. FERMI: A Stochastic Distributionally Robust Fair Empirical Risk Minimization Framework](https://mlsanity.com/paper/2309.11682/) (2023)
- [Effective-LDAM: An Effective Loss Function To Mitigate Data Imbalance for Robust Chest X-Ray Disease Classification](https://mlsanity.com/paper/2407.04953/) (2024)
- [Bayesian analysis of the prevalence bias: learning and predicting from imbalanced data](https://mlsanity.com/paper/2108.00250/) (2021)
- [Test-Time Mixup Augmentation for Data and Class-Specific Uncertainty Estimation in Deep Learning Image Classification](https://mlsanity.com/paper/2212.00214/) (2022)
- [Optimized Hybrid Focal Margin Loss for Crack Segmentation](https://mlsanity.com/paper/2302.04395/) (2023)
- [On the Impact of Class Imbalance on the Learning Dynamics of Deep Neural Networks:An Intuitive Insight](https://mlsanity.com/paper/2605.24908/) (2026)

## Abstract (arXiv)

Overfitting in deep learning has been the focus of a number of recent works, yet its exact impact on the behavior of neural networks is not well understood. This study analyzes overfitting by examining how the distribution of logits alters in relation to how much the model overfits. Specifically, we find that when training with few data samples, the distribution of logit activations when processing unseen test samples of an under-represented class tends to shift towards and even across the decision boundary, while the over-represented class seems unaffected. In image segmentation, foreground samples are often heavily under-represented. We observe that sensitivity of the model drops as a result of overfitting, while precision remains mostly stable. Based on our analysis, we derive asymmetric modifications of existing loss functions and regularizers including a large margin loss, focal loss, adversarial training and mixup, which specifically aim at reducing the shift observed when embedding unseen samples of the under-represented class. We study the case of binary segmentation of brain tumor core and show that our proposed simple modifications lead to significantly improved segmentation performance over the symmetric variants.