mlsanityData to 7 Oct 2026

Overfitting of neural nets under class imbalance: Analysis and improvements for segmentation

arXiv 1907.10982 ↗First posted 25 Jul 2019cs.LG cs.CV stat.MLcited by 27

Not on the map. Why?

Abstract (arXiv)

Overfitting in deep learning has been the focus of a number of recent works, yet its exact impact on the behavior of neural networks is not well understood. This study analyzes overfitting by examining how the distribution of logits alters in relation to how much the model overfits. Specifically, we find that when training with few data samples, the distribution of logit activations when processing unseen test samples of an under-represented class tends to shift towards and even across the decision boundary, while the over-represented class seems unaffected. In image segmentation, foreground samples are often heavily under-represented. We observe that sensitivity of the model drops as a result of overfitting, while precision remains mostly stable. Based on our analysis, we derive asymmetric modifications of existing loss functions and regularizers including a large margin loss, focal loss, adversarial training and mixup, which specifically aim at reducing the shift observed when embedding unseen samples of the under-represented class. We study the case of binary segmentation of brain tumor core and show that our proposed simple modifications lead to significantly improved segmentation performance over the symmetric variants.

Key facts. Cited by 27 papers in the map (data through 7 Oct 2026). First posted on arXiv 25 Jul 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 8; 2022: 6; 2023: 8; 2024: 3; 2025: 1; 2026: 1.

27 papers.

Sort:

By area:

  1. 2021Unified Focal loss: Generalising Dice and cross entropy-based losses to handle class imbalanced medical image segmentationcited by 88
  2. 2023Conditional Diffusion Models for Semantic 3D Brain MRI Synthesiscited by 85
  3. 2023Rethinking Semi-Supervised Medical Image Segmentation: A Variance-Reduction Perspectivecited by 43
  4. 2021Analyzing Overfitting under Class Imbalance in Neural Networks for Image Segmentationcited by 34
  5. 2021Deep learning-based synthetic-CT generation in radiotherapy and PET: a reviewcited by 28
  6. 2021Enhancing MR Image Segmentation with Realistic Adversarial Data Augmentationcited by 20
  7. 2021Constrained Optimization to Train Neural Networks on Critical and Under-Represented Classescited by 19
  8. 2023InPL: Pseudo-labeling the Inliers First for Imbalanced Semi-supervised Learningcited by 12
  9. 2022VisCUIT: Visual Auditor for Bias in CNN Image Classifiercited by 8
  10. 2022RESECT-SEG: Open access annotations of intra-operative brain tumor ultrasound imagescited by 6
  11. 2022Joint Modeling of Image and Label Statistics for Enhancing Model Generalizability of Medical Image Segmentationcited by 5
  12. 2023Co-Learning Meets Stitch-Up for Noisy Multi-label Visual Recognitioncited by 4
15 more
  1. 2021Detecting, Localising and Classifying Polyps from Colonoscopy Videos using Deep Learningcited by 3
  2. 2025Balancing Multi-Target Semi-Supervised Medical Image Segmentation with Collaborative Generalist and Specialistscited by 3
  3. 2024Transesophageal Echocardiography Generation using Anatomical Modelscited by 2
  4. 2024On Discriminative Probabilistic Modeling for Self-Supervised Representation Learningcited by 2
  5. 2021MetaBalance: High-Performance Neural Networks for Class-Imbalanced Datacited by 1
  6. 2022Empirical Study of Overfitting in Deep FNN Prediction Models for Breast Cancer Metastasiscited by 1
  7. 2022Context Label Learning: Improving Background Class Representations in Semantic Segmentationcited by 1
  8. 2023Ranking Regularization for Critical Rare Classes: Minimizing False Positives at a High True Positive Ratecited by 1
  9. 2023The Effects of Mixed Sample Data Augmentation are Class Dependentcited by 1
  10. 2023Dr. FERMI: A Stochastic Distributionally Robust Fair Empirical Risk Minimization Frameworkcited by 1
  11. 2024Effective-LDAM: An Effective Loss Function To Mitigate Data Imbalance for Robust Chest X-Ray Disease Classificationcited by 1
  12. 2021Bayesian analysis of the prevalence bias: learning and predicting from imbalanced datacited by 0
  13. 2022Test-Time Mixup Augmentation for Data and Class-Specific Uncertainty Estimation in Deep Learning Image Classificationcited by 0
  14. 2023Optimized Hybrid Focal Margin Loss for Crack Segmentationcited by 0
  15. 2026On the Impact of Class Imbalance on the Learning Dynamics of Deep Neural Networks:An Intuitive Insightcited by 0