# Enhancing Label Correlation Feedback in Multi-Label Text Classification via Multi-Task Learning

arXiv: https://arxiv.org/abs/2106.03103 · HTML: https://mlsanity.com/paper/2106.03103/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 37 references that are in the map.
- First posted on arXiv 6 Jun 2021.

## Builds on (most cited first, 16 of 16)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Convolutional Neural Networks for Sentence Classification](https://mlsanity.com/paper/1408.5882/) (2014)
- [Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks](https://mlsanity.com/paper/1506.03099/) (2015)
- [Long-tail learning via logit adjustment](https://mlsanity.com/paper/2007.07314/) (2020)
- [Bidirectional LSTM-CRF Models for Sequence Tagging](https://mlsanity.com/paper/1508.01991/) (2015)
- [Order Matters: Sequence to sequence for sets](https://mlsanity.com/paper/1511.06391/) (2015)
- [Recurrent Neural Network for Text Classification with Multi-Task Learning](https://mlsanity.com/paper/1605.05101/) (2016)
- [AttentionXML: Label Tree-based Attention-Aware Deep Model for High-Performance Extreme Multi-Label Text Classification](https://mlsanity.com/paper/1811.01727/) (2018)
- [The Emerging Trends of Multi-Label Learning](https://mlsanity.com/paper/2011.11197/) (2020)
- [SGM: Sequence Generation Model for Multi-label Classification](https://mlsanity.com/paper/1806.04822/) (2018)
- [Joint Embedding of Words and Labels for Text Classification](https://mlsanity.com/paper/1805.04174/) (2018)
- [DocBERT: BERT for Document Classification](https://mlsanity.com/paper/1904.08398/) (2019)
- [Label Confusion Learning to Enhance Text Classification Models](https://mlsanity.com/paper/2012.04987/) (2020)
- [Order-free Learning Alleviating Exposure Bias in Multi-label Classification](https://mlsanity.com/paper/1909.03434/) (2019)
- [Adapting RNN Sequence Prediction Model to Multi-label Set Prediction](https://mlsanity.com/paper/1904.05829/) (2019)

## Built on by (most cited first, 15 of 15)

- [CrisisSense-LLM: Instruction Fine-Tuned Large Language Model for Multi-label Social Media Text Classification in Disaster Informatics](https://mlsanity.com/paper/2406.15477/) (2024)
- [An Effective Deployment of Contrastive Learning in Multi-label Text Classification](https://mlsanity.com/paper/2212.00552/) (2022)
- [Multi-Label Classification for Implicit Discourse Relation Recognition](https://mlsanity.com/paper/2406.04461/) (2024)
- [Exploring Contrastive Learning for Long-Tailed Multi-Label Text Classification](https://mlsanity.com/paper/2404.08720/) (2024)
- [Multi-Label Contrastive Learning : A Comprehensive Study](https://mlsanity.com/paper/2412.00101/) (2024)
- [Adversarial Ensemble Training by Jointly Learning Label Dependencies and Member Models](https://mlsanity.com/paper/2206.14477/) (2022)
- [Bike Frames: Understanding the Implicit Portrayal of Cyclists in the News](https://mlsanity.com/paper/2301.06178/) (2023)
- [Cluster-Guided Label Generation in Extreme Multi-Label Classification](https://mlsanity.com/paper/2302.09150/) (2023)
- [Accurate Use of Label Dependency in Multi-Label Text Classification Through the Lens of Causality](https://mlsanity.com/paper/2310.07588/) (2023)
- [Pseudo-Prompt Generating in Pre-trained Vision-Language Models for Multi-Label Medical Image Classification](https://mlsanity.com/paper/2405.06468/) (2024)
- [LabelCoRank: Revolutionizing Long Tail Multi-Label Classification with Co-Occurrence Reranking](https://mlsanity.com/paper/2503.07968/) (2025)
- [Multi-Label Node Classification with Label Influence Propagation](https://mlsanity.com/paper/2607.00671/) (2026)
- [Implicit Training of Energy Model for Structure Prediction](https://mlsanity.com/paper/2211.11649/) (2022)
- [A Debiased Nearest Neighbors Framework for Multi-Label Text Classification](https://mlsanity.com/paper/2408.03202/) (2024)
- [University of Indonesia at SemEval-2025 Task 11: Evaluating State-of-the-Art Encoders for Multi-Label Emotion Detection](https://mlsanity.com/paper/2505.16460/) (2025)

## Abstract (arXiv)

In multi-label text classification (MLTC), each given document is associated with a set of correlated labels. To capture label correlations, previous classifier-chain and sequence-to-sequence models transform MLTC to a sequence prediction task. However, they tend to suffer from label order dependency, label combination over-fitting and error propagation problems. To address these problems, we introduce a novel approach with multi-task learning to enhance label correlation feedback. We first utilize a joint embedding (JE) mechanism to obtain the text and label representation simultaneously. In MLTC task, a document-label cross attention (CA) mechanism is adopted to generate a more discriminative document representation. Furthermore, we propose two auxiliary label co-occurrence prediction tasks to enhance label correlation learning: 1) Pairwise Label Co-occurrence Prediction (PLCP), and 2) Conditional Label Co-occurrence Prediction (CLCP). Experimental results on AAPD and RCV1-V2 datasets show that our method outperforms competitive baselines by a large margin. We analyze low-frequency label performance, label dependency, label combination diversity and coverage speed to show the effectiveness of our proposed method on label correlation learning.