mlsanityData to 7 Oct 2026

Balancing Methods for Multi-label Text Classification with Long-Tailed Class Distribution

arXiv 2109.04712 ↗First posted 10 Sep 2021cs.CLcited by 34

Not on the map. Why?

Abstract (arXiv)

Multi-label text classification is a challenging task because it requires capturing label dependencies. It becomes even more challenging when class distribution is long-tailed. Resampling and re-weighting are common approaches used for addressing the class imbalance problem, however, they are not effective when there is label dependency besides class imbalance because they result in oversampling of common labels. Here, we introduce the application of balancing loss functions for multi-label text classification. We perform experiments on a general domain dataset with 90 labels (Reuters-21578) and a domain-specific dataset from PubMed with 18211 labels. We find that a distribution-balanced loss function, which inherently addresses both the class imbalance and label linkage problems, outperforms commonly used loss functions. Distribution balancing methods have been successfully used in the image recognition field. Here, we show their effectiveness in natural language processing. Source code is available at https://github.com/Roche/BalancedLossNLP.

Key facts. Cited by 34 papers in the map (data through 7 Oct 2026). Builds on 10 of its 30 references that are in the map. First posted on arXiv 10 Sep 2021.

Builds on

10 of 30 references.

References · 10

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2016V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentationcited by 2,284
  3. 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,935
  4. 2019Class-Balanced Loss Based on Effective Number of Samplescited by 1,239
  5. 2020SPECTER: Document-level Representation Learning using Citation-informed Transformerscited by 426
  6. 2019Dice Loss for Data-imbalanced NLP Taskscited by 180
  7. 2019Learning a Deep ConvNet for Multi-label Classification with Partial Labelscited by 124
  8. 2020Distribution-Balanced Loss for Multi-Label Classification in Long-Tailed Datasetscited by 122
  9. 2019Learning to Learn and Predict: A Meta-Learning Approach for Multi-Label Classificationcited by 35
  10. 2019Order-free Learning Alleviating Exposure Bias in Multi-label Classificationcited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 5; 2023: 9; 2024: 6; 2025: 8; 2026: 6.

34 papers.

Sort:

  1. 2022A Survey of Methods for Addressing Class Imbalance in Deep-Learning Based Natural Language Processingcited by 20
  2. 2023Neural Additive Models for Location Scale and Shape: A Framework for Interpretable Neural Regression Beyond the Meancited by 18
  3. 2023Compositional Generalization for Multi-label Text Classification: A Data-Augmentation Approachcited by 7
  4. 2023Recent Advances in Hierarchical Multi-label Text Classification: A Surveycited by 6
  5. 2024Multimodal Metadata Assignment for Cultural Heritage Artifactscited by 5
  6. 2024Multi-Stage Balanced Distillation: Addressing Long-Tail Challenges in Sequence-Level Knowledge Distillationcited by 5
  7. 2023SEPSIS: I Can Catch Your Lies -- A New Paradigm for Deception Detectioncited by 4
  8. 2025Curiosity Meets Cooperation: A Game-Theoretic Approach to Long-Tail Multi-Label Learningcited by 4
  9. 2024Konstruktor: A Strong Baseline for Simple Knowledge Graph Question Answeringcited by 3
  10. 2025Task-Informed Anti-Curriculum by Masking Improves Downstream Performance on Textcited by 3
  11. 2025Label-semantics Aware Generative Approach for Domain-Agnostic Multilabel Classificationcited by 2
  12. 2025Fairness Metric Design Exploration in Multi-Domain Moral Sentiment Classification using Transformer-Based Modelscited by 2
22 more
  1. 2022Are We Really Making Much Progress in Text Classification? A Comparative Reviewcited by 1
  2. 2022ViralBERT: A User Focused BERT-Based Approach to Virality Predictioncited by 1
  3. 2023DOST -- Domain Obedient Self-supervised Training for Multi Label Classification with Noisy Labelscited by 1
  4. 2023Text2Tree: Aligning Text Representation to the Label Tree Hierarchy for Imbalanced Medical Classificationcited by 1
  5. 2024LEGION: Harnessing Pre-trained Language Models for GitHub Topic Recommendations with Distribution-Balance Losscited by 1
  6. 2024Empowering Interdisciplinary Research with BERT-Based Models: An Approach Through SciBERT-CNN with Topic Modelingcited by 1
  7. 2025Towards the Development of Balanced Synthetic Data for Correcting Grammatical Errors in Arabic: An Approach Based on Error Tagging Model and Synthetic Data Generating Modelcited by 1
  8. 2025LabelCoRank: Revolutionizing Long Tail Multi-Label Classification with Co-Occurrence Rerankingcited by 1
  9. 2022A Comparison of Approaches for Imbalanced Classification Problems in the Context of Retrieving Relevant Documents for an Analysiscited by 0
  10. 2022Harnessing label semantics to extract higher performance under noisy label for Company to Industry matchingcited by 0
  11. 2023Large-scale investigation of weakly-supervised deep learning for the fine-grained semantic indexing of biomedical literaturecited by 0
  12. 2023APAM: Adaptive Pre-training and Adaptive Meta Learning in Language Model for Noisy Labels and Long-tailed Learningcited by 0
  13. 2023Scalable Label Distribution Learning for Multi-Label Classificationcited by 0
  14. 2024A Debiased Nearest Neighbors Framework for Multi-Label Text Classificationcited by 0
  15. 2025Enhancing Automatic PT Tagging for MEDLINE Citations Using Transformer-Based Modelscited by 0
  16. 2025Causal Invariance and Counterfactual Learning Driven Cooperative Game for Multi-Label Classificationcited by 0
  17. 2026A Scalable Curiosity-Driven Game-Theoretic Framework for Long-Tail Multi-Label Learning in Data Miningcited by 0
  18. 2026Human Label Variation in Implicit Discourse Relation Recognitioncited by 0
  19. 2026Semi-Supervised Learning with Balanced Deep Representation Distributionscited by 0
  20. 2026AstroConcepts: A Large-Scale Multi-Label Classification Corpus for Astrophysicscited by 0
  21. 2026MADE: A Living Benchmark for Multi-Label Text Classification with Uncertainty Quantification of Medical Device Adverse Eventscited by 0
  22. 2026Automatic Thematic Indexing of Large Literary Corpora: A Machine Learning Approach to Voltaire's Complete Workscited by 0