# A Survey of Methods for Addressing Class Imbalance in Deep-Learning Based Natural Language Processing

arXiv: https://arxiv.org/abs/2210.04675 · HTML: https://mlsanity.com/paper/2210.04675/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 66 of its 123 references that are in the map.
- First posted on arXiv 10 Oct 2022.

## Builds on (most cited first, 30 of 66)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features](https://mlsanity.com/paper/1905.04899/) (2019)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/1606.04797/) (2016)
- [Don't Stop Pretraining: Adapt Language Models to Domains and Tasks](https://mlsanity.com/paper/2004.10964/) (2020)
- [LVIS: A Dataset for Large Vocabulary Instance Segmentation](https://mlsanity.com/paper/1908.03195/) (2019)
- [Convolutional Neural Networks for Sentence Classification](https://mlsanity.com/paper/1408.5882/) (2014)
- [Unsupervised Data Augmentation for Consistency Training](https://mlsanity.com/paper/1904.12848/) (2019)
- [Learning Imbalanced Datasets with Label-Distribution-Aware Margin Loss](https://mlsanity.com/paper/1906.07413/) (2019)
- [Decoupling Representation and Classifier for Long-Tailed Recognition](https://mlsanity.com/paper/1910.09217/) (2019)
- [EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Tasks](https://mlsanity.com/paper/1901.11196/) (2019)
- [MINE: Mutual Information Neural Estimation](https://mlsanity.com/paper/1801.04062/) (2018)
- [A systematic study of the class imbalance problem in convolutional neural networks](https://mlsanity.com/paper/1710.05381/) (2017)
- [Deep Long-Tailed Learning: A Survey](https://mlsanity.com/paper/2110.04596/) (2021)
- [A Survey of Data Augmentation Approaches for NLP](https://mlsanity.com/paper/2105.03075/) (2021)
- [HateXplain: A Benchmark Dataset for Explainable Hate Speech Detection](https://mlsanity.com/paper/2012.10289/) (2020)
- [Frustratingly Simple Few-Shot Object Detection](https://mlsanity.com/paper/2003.06957/) (2020)
- [Massively Multilingual Neural Machine Translation in the Wild: Findings and Challenges](https://mlsanity.com/paper/1907.05019/) (2019)
- [Explainable Prediction of Medical Codes from Clinical Text](https://mlsanity.com/paper/1802.05695/) (2018)
- [Distribution Alignment: A Unified Framework for Long-tail Visual Recognition](https://mlsanity.com/paper/2103.16370/) (2021)
- [Dice Loss for Data-imbalanced NLP Tasks](https://mlsanity.com/paper/1911.02855/) (2019)
- [A Survey on Recent Approaches for Natural Language Processing in Low-Resource Scenarios](https://mlsanity.com/paper/2010.12309/) (2020)
- [Delving into Deep Imbalanced Regression](https://mlsanity.com/paper/2102.09554/) (2021)
- [Neural Unsupervised Domain Adaptation in NLP---A Survey](https://mlsanity.com/paper/2006.00632/) (2020)
- [Self-supervised Learning is More Robust to Dataset Imbalance](https://mlsanity.com/paper/2110.05025/) (2021)
- [Distribution-Balanced Loss for Multi-Label Classification in Long-Tailed Datasets](https://mlsanity.com/paper/2007.09654/) (2020)
- [Relay Backpropagation for Effective Learning of Deep Convolutional Neural Networks](https://mlsanity.com/paper/1512.05830/) (2015)

## Built on by (most cited first, 21 of 21)

- [Fill-Up: Balancing Long-Tailed Data with Generative Models](https://mlsanity.com/paper/2306.07200/) (2023)
- [Class-Imbalanced Learning on Graphs: A Survey](https://mlsanity.com/paper/2304.04300/) (2023)
- [Paraphrase Identification with Deep Learning: A Review of Datasets and Methods](https://mlsanity.com/paper/2212.06933/) (2022)
- [What makes a good metric? Evaluating automatic metrics for text-to-image consistency](https://mlsanity.com/paper/2412.13989/) (2024)
- [A Comprehensive Survey on Imbalanced Data Learning](https://mlsanity.com/paper/2502.08960/) (2025)
- [Large Language Models, scientific knowledge and factuality: A framework to streamline human expert evaluation](https://mlsanity.com/paper/2305.17819/) (2023)
- [Examining Imbalance Effects on Performance and Demographic Fairness of Clinical Language Models](https://mlsanity.com/paper/2412.17803/) (2024)
- [MuLMS-AZ: An Argumentative Zoning Dataset for the Materials Science Domain](https://mlsanity.com/paper/2307.02340/) (2023)
- [DWReCO at CheckThat! 2023: Enhancing Subjectivity Detection through Style-based Data Sampling](https://mlsanity.com/paper/2307.03550/) (2023)
- [Adapting Language Balance in Code-Switching Speech](https://mlsanity.com/paper/2510.18724/) (2025)
- [Reshaping Free-Text Radiology Notes Into Structured Reports With Generative Transformers](https://mlsanity.com/paper/2403.18938/) (2024)
- [Deep Dive into the Language of International Relations: NLP-based Analysis of UNESCO's Summary Records](https://mlsanity.com/paper/2307.16573/) (2023)
- [IAI Group at CheckThat! 2024: Transformer Models and Data Augmentation for Checkworthy Claim Detection](https://mlsanity.com/paper/2408.01118/) (2024)
- [Class-Aware Contrastive Optimization for Imbalanced Text Classification](https://mlsanity.com/paper/2410.22197/) (2024)
- [Active Learning Methods for Efficient Data Utilization and Model Performance Enhancement](https://mlsanity.com/paper/2504.16136/) (2025)
- [Extrapolated Markov Chain Oversampling Method for Imbalanced Text Classification](https://mlsanity.com/paper/2509.02332/) (2025)
- [Modeling Human-Like Color Naming Behavior in Context](https://mlsanity.com/paper/2604.25674/) (2026)
- [AutoTail-BSFGM: Class-Balance-Aware Fine-Tuning for Chinese Scholarly Text Classification](https://mlsanity.com/paper/2606.03576/) (2026)
- [Ensemble of Unsupervised Deep Learning for Clustering Imbalanced Tabular Data](https://mlsanity.com/paper/2608.00346/) (2026)
- [Subjective Multi-Bias Detection with Large Language Models](https://mlsanity.com/paper/2608.09126/) (2026)
- [Fine-Grained Emotion Classification from Mobile App Reviews: An Empirical Study with Large Language Models](https://mlsanity.com/paper/2610.03802/) (2026)

## Abstract (arXiv)

Many natural language processing (NLP) tasks are naturally imbalanced, as some target categories occur much more frequently than others in the real world. In such scenarios, current NLP models still tend to perform poorly on less frequent classes. Addressing class imbalance in NLP is an active research topic, yet, finding a good approach for a particular task and imbalance scenario is difficult. With this survey, the first overview on class imbalance in deep-learning based NLP, we provide guidance for NLP researchers and practitioners dealing with imbalanced data. We first discuss various types of controlled and real-world class imbalance. Our survey then covers approaches that have been explicitly proposed for class-imbalanced NLP tasks or, originating in the computer vision community, have been evaluated on them. We organize the methods by whether they are based on sampling, data augmentation, choice of loss function, staged learning, or model design. Finally, we discuss open problems such as dealing with multi-label scenarios, and propose systematic benchmarking and reporting in order to move forward on this problem as a community.