# On the combined effect of class imbalance and concept complexity in deep learning

arXiv: https://arxiv.org/abs/2107.14194 · HTML: https://mlsanity.com/paper/2107.14194/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 25 references that are in the map.
- First posted on arXiv 29 Jul 2021.

## Builds on (most cited first, 9 of 9)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Rich feature hierarchies for accurate object detection and semantic segmentation](https://mlsanity.com/paper/1311.2524/) (2013)
- [Learning Imbalanced Datasets with Label-Distribution-Aware Margin Loss](https://mlsanity.com/paper/1906.07413/) (2019)
- [Deep Learning on Graphs: A Survey](https://mlsanity.com/paper/1812.04202/) (2018)
- [Distribution-Balanced Loss for Multi-Label Classification in Long-Tailed Datasets](https://mlsanity.com/paper/2007.09654/) (2020)
- [Generative Adversarial Minority Oversampling](https://mlsanity.com/paper/1903.09730/) (2019)
- [DeepSMOTE: Fusing Deep Learning and SMOTE for Imbalanced Data](https://mlsanity.com/paper/2105.02340/) (2021)
- [Class-Incremental Experience Replay for Continual Learning under Concept Drift](https://mlsanity.com/paper/2104.11861/) (2021)

## Built on by (most cited first, 2 of 2)

- [Efficient Augmentation for Imbalanced Deep Learning](https://mlsanity.com/paper/2207.06080/) (2022)
- [Understanding the Challenges When 3D Semantic Segmentation Faces Class Imbalanced and OOD Data](https://mlsanity.com/paper/2203.00214/) (2022)

## Abstract (arXiv)

Structural concept complexity, class overlap, and data scarcity are some of the most important factors influencing the performance of classifiers under class imbalance conditions. When these effects were uncovered in the early 2000s, understandably, the classifiers on which they were demonstrated belonged to the classical rather than Deep Learning categories of approaches. As Deep Learning is gaining ground over classical machine learning and is beginning to be used in critical applied settings, it is important to assess systematically how well they respond to the kind of challenges their classical counterparts have struggled with in the past two decades. The purpose of this paper is to study the behavior of deep learning systems in settings that have previously been deemed challenging to classical machine learning systems to find out whether the depth of the systems is an asset in such settings. The results in both artificial and real-world image datasets (MNIST Fashion, CIFAR-10) show that these settings remain mostly challenging for Deep Learning systems and that deeper architectures seem to help with structural concept complexity but not with overlap challenges in simple artificial domains. Data scarcity is not overcome by deeper layers, either. In the real-world image domains, where overfitting is a greater concern than in the artificial domains, the advantage of deeper architectures is less obvious: while it is observed in certain cases, it is quickly cancelled as models get deeper and perform worse than their shallower counterparts.