mlsanityData to 14 Oct 2026

A Survey of Methods for Addressing Class Imbalance in Deep-Learning Based Natural Language Processing

arXiv 2210.04675 ↗First posted 10 Oct 2022cs.CL cs.AIcited by 21

Not on the map. Why?

Abstract (arXiv)

Many natural language processing (NLP) tasks are naturally imbalanced, as some target categories occur much more frequently than others in the real world. In such scenarios, current NLP models still tend to perform poorly on less frequent classes. Addressing class imbalance in NLP is an active research topic, yet, finding a good approach for a particular task and imbalance scenario is difficult. With this survey, the first overview on class imbalance in deep-learning based NLP, we provide guidance for NLP researchers and practitioners dealing with imbalanced data. We first discuss various types of controlled and real-world class imbalance. Our survey then covers approaches that have been explicitly proposed for class-imbalanced NLP tasks or, originating in the computer vision community, have been evaluated on them. We organize the methods by whether they are based on sampling, data augmentation, choice of loss function, staged learning, or model design. Finally, we discuss open problems such as dealing with multi-label scenarios, and propose systematic benchmarking and reporting in order to move forward on this problem as a community.

Key facts. Cited by 21 papers in the map (data through 14 Oct 2026). Builds on 66 of its 123 references that are in the map. First posted on arXiv 10 Oct 2022.

Builds on

66 of 123 references.

References · 66

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  4. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,474
  5. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,884
  6. 2020Supervised Contrastive Learningcited by 2,829
  7. 2016V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentationcited by 2,292
  8. 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,565
  9. 2019LVIS: A Dataset for Large Vocabulary Instance Segmentationcited by 1,217
  10. 2014Convolutional Neural Networks for Sentence Classificationcited by 1,127
  11. 2019Unsupervised Data Augmentation for Consistency Trainingcited by 1,060
  12. 2019Learning Imbalanced Datasets with Label-Distribution-Aware Margin Losscited by 989
38 more
  1. 2019Decoupling Representation and Classifier for Long-Tailed Recognitioncited by 809
  2. 2019EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Taskscited by 807
  3. 2018MINE: Mutual Information Neural Estimationcited by 683
  4. 2017A systematic study of the class imbalance problem in convolutional neural networkscited by 513
  5. 2021Deep Long-Tailed Learning: A Surveycited by 417
  6. 2021A Survey of Data Augmentation Approaches for NLPcited by 409
  7. 2020HateXplain: A Benchmark Dataset for Explainable Hate Speech Detectioncited by 407
  8. 2020Frustratingly Simple Few-Shot Object Detectioncited by 282
  9. 2019Massively Multilingual Neural Machine Translation in the Wild: Findings and Challengescited by 257
  10. 2018Explainable Prediction of Medical Codes from Clinical Textcited by 234
  11. 2021Distribution Alignment: A Unified Framework for Long-tail Visual Recognitioncited by 198
  12. 2019Dice Loss for Data-imbalanced NLP Taskscited by 182
  13. 2020A Survey on Recent Approaches for Natural Language Processing in Low-Resource Scenarioscited by 171
  14. 2021Delving into Deep Imbalanced Regressioncited by 158
  15. 2020Neural Unsupervised Domain Adaptation in NLP---A Surveycited by 144
  16. 2021Self-supervised Learning is More Robust to Dataset Imbalancecited by 123
  17. 2020Distribution-Balanced Loss for Multi-Label Classification in Long-Tailed Datasetscited by 122
  18. 2015Relay Backpropagation for Effective Learning of Deep Convolutional Neural Networkscited by 115
  19. 2020The Devil is in Classification: A Simple Framework for Long-tail Object Detection and Instance Segmentationcited by 108
  20. 2021Measure and Improve Robustness in NLP Models: A Surveycited by 107
  21. 2021An Improved Baseline for Sentence-level Relation Extractioncited by 81
  22. 2019Long-tail Relation Extraction via Knowledge Graph Embeddings and Graph Convolution Networkscited by 61
  23. 2021Swiss-Judgment-Prediction: A Multilingual Legal Judgment Prediction Benchmarkcited by 60
  24. 2020A Survey of Active Learning for Text Classification using Deep Neural Networkscited by 50
  25. 2021Not All Negatives are Equal: Label-Aware Contrastive Loss for Fine-grained Text Classificationcited by 43
  26. 2021True Few-Shot Learning with Prompts -- A Real-World Perspectivecited by 38
  27. 2020Adapt-and-Adjust: Overcoming the Long-Tail Problem of Multilingual Speech Recognitioncited by 35
  28. 2021Balancing Methods for Multi-label Text Classification with Long-Tailed Class Distributioncited by 35
  29. 2015Classifying Relations by Ranking with Convolutional Neural Networkscited by 33
  30. 2021Memorisation versus Generalisation in Pre-trained Language Modelscited by 32
  31. 2021Uncovering Main Causalities for Long-tailed Information Extractioncited by 21
  32. 2021Fairness-aware Class Imbalanced Learningcited by 19
  33. 2017Boosted Cascaded Convnets for Multilabel Classification of Thoracic Diseases in Chest Radiographscited by 18
  34. 2020Cost-Sensitive BERT for Generalisable Sentence Classification with Imbalanced Datacited by 14
  35. 2021TransICD: Transformer Based Code-wise Attention Model for Explainable ICD Codingcited by 13
  36. 2020A little goes a long way: Improving toxic language classification despite data scarcitycited by 12
  37. 2020Multi-Label Sentiment Analysis on 100 Languages with Dynamic Weighting for Label Imbalancecited by 11
  38. 2020Supertagging the Long Tail with Tree-Structured Decoding of Complex Categoriescited by 11

50 most cited of 66 All 66 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 1; 2023: 6; 2024: 5; 2025: 4; 2026: 5.

21 papers.

Sort:

  1. 2023Fill-Up: Balancing Long-Tailed Data with Generative Modelscited by 34
  2. 2023Class-Imbalanced Learning on Graphs: A Surveycited by 24
  3. 2022Paraphrase Identification with Deep Learning: A Review of Datasets and Methodscited by 13
  4. 2024What makes a good metric? Evaluating automatic metrics for text-to-image consistencycited by 12
  5. 2025A Comprehensive Survey on Imbalanced Data Learningcited by 10
  6. 2023Large Language Models, scientific knowledge and factuality: A framework to streamline human expert evaluationcited by 4
  7. 2024Examining Imbalance Effects on Performance and Demographic Fairness of Clinical Language Modelscited by 4
  8. 2023MuLMS-AZ: An Argumentative Zoning Dataset for the Materials Science Domaincited by 2
  9. 2023DWReCO at CheckThat! 2023: Enhancing Subjectivity Detection through Style-based Data Samplingcited by 2
  10. 2025Adapting Language Balance in Code-Switching Speechcited by 2
  11. 2024Reshaping Free-Text Radiology Notes Into Structured Reports With Generative Transformerscited by 1
  12. 2023Deep Dive into the Language of International Relations: NLP-based Analysis of UNESCO's Summary Recordscited by 0
9 more
  1. 2024IAI Group at CheckThat! 2024: Transformer Models and Data Augmentation for Checkworthy Claim Detectioncited by 0
  2. 2024Class-Aware Contrastive Optimization for Imbalanced Text Classificationcited by 0
  3. 2025Active Learning Methods for Efficient Data Utilization and Model Performance Enhancementcited by 0
  4. 2025Extrapolated Markov Chain Oversampling Method for Imbalanced Text Classificationcited by 0
  5. 2026Modeling Human-Like Color Naming Behavior in Contextcited by 0
  6. 2026AutoTail-BSFGM: Class-Balance-Aware Fine-Tuning for Chinese Scholarly Text Classificationcited by 0
  7. 2026Ensemble of Unsupervised Deep Learning for Clustering Imbalanced Tabular Datacited by 0
  8. 2026Subjective Multi-Bias Detection with Large Language Modelscited by 0
  9. 2026Fine-Grained Emotion Classification from Mobile App Reviews: An Empirical Study with Large Language Modelscited by 0