# An Effective Deployment of Contrastive Learning in Multi-label Text Classification arXiv: https://arxiv.org/abs/2212.00552 ยท HTML: https://mlsanity.com/paper/2212.00552/ ## Key facts - Cited by 9 papers in the map (data through 14 Oct 2026). - Builds on 25 of its 60 references that are in the map. - First posted on arXiv 1 Dec 2022. ## Builds on (most cited first, 25 of 25) - [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020) - [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021) - [Asymmetric Loss For Multi-Label Classification](https://mlsanity.com/paper/2009.14119/) (2020) - [Supervised Contrastive Learning for Pre-trained Language Model Fine-tuning](https://mlsanity.com/paper/2011.01403/) (2020) - [Crowdsourcing a Word-Emotion Association Lexicon](https://mlsanity.com/paper/1308.6297/) (2013) - [SGM: Sequence Generation Model for Multi-label Classification](https://mlsanity.com/paper/1806.04822/) (2018) - [KUISAIL at SemEval-2020 Task 12: BERT-CNN for Offensive Speech Identification in Social Media](https://mlsanity.com/paper/2007.13184/) (2020) - [ERICA: Improving Entity and Relation Understanding for Pre-trained Language Models via Contrastive Learning](https://mlsanity.com/paper/2012.15022/) (2020) - [RealFormer: Transformer Likes Residual Attention](https://mlsanity.com/paper/2012.11747/) (2020) - [Use All The Labels: A Hierarchical Multi-Label Contrastive Learning Framework](https://mlsanity.com/paper/2204.13207/) (2022) - [SpanEmo: Casting Multi-label Emotion Classification as Span-prediction](https://mlsanity.com/paper/2101.10038/) (2021) - [Multi-Label Contrastive Learning for Abstract Visual Reasoning](https://mlsanity.com/paper/2012.01944/) (2020) - [NTUA-SLP at SemEval-2018 Task 1: Predicting Affective Content in Tweets with Deep Attentive RNNs and Transfer Learning](https://mlsanity.com/paper/1804.06658/) (2018) - [Improved Multi-label Classification under Temporal Concept Drift: Rethinking Group-Robust Algorithms in a Label-Wise Setting](https://mlsanity.com/paper/2203.07856/) (2022) - [SemEval-2013 Task 2: Sentiment Analysis in Twitter](https://mlsanity.com/paper/1912.06806/) (2019) - [Multi-Label Image Classification with Contrastive Learning](https://mlsanity.com/paper/2107.11626/) (2021) - [Enhancing Label Correlation Feedback in Multi-Label Text Classification via Multi-Task Learning](https://mlsanity.com/paper/2106.03103/) (2021) - [Gaussian Mixture Variational Autoencoder with Contrastive Learning for Multi-Label Classification](https://mlsanity.com/paper/2112.00976/) (2021) - [Joint Binary Neural Network for Multi-label Learning with Applications to Emotion Classification](https://mlsanity.com/paper/1802.00891/) (2018) - [EmoGraph: Capturing Emotion Correlations using Graph Networks](https://mlsanity.com/paper/2008.09378/) (2020) - [Multi-Label Sentiment Analysis on 100 Languages with Dynamic Weighting for Label Imbalance](https://mlsanity.com/paper/2008.11573/) (2020) - [Semantic-Unit-Based Dilated Convolution for Multi-Label Text Classification](https://mlsanity.com/paper/1808.08561/) (2018) - [LiGCN: Label-interpretable Graph Convolutional Networks for Multi-label Text Classification](https://mlsanity.com/paper/2103.14620/) (2021) - [Happy Dance, Slow Clap: Using Reaction GIFs to Predict Induced Affect on Twitter](https://mlsanity.com/paper/2105.09967/) (2021) - [A Dual-Contrastive Framework for Low-Resource Cross-Lingual Named Entity Recognition](https://mlsanity.com/paper/2204.00796/) (2022) ## Built on by (most cited first, 9 of 9) - [Leveraging Label Semantics and Meta-Label Refinement for Multi-Label Question Classification](https://mlsanity.com/paper/2411.01841/) (2024) - [ProtoECGNet: Case-Based Interpretable Deep Learning for Multi-Label ECG Classification with Contrastive Learning](https://mlsanity.com/paper/2504.08713/) (2025) - [Analyzing Persuasive Strategies in Meme Texts: A Fusion of Language Models with Paraphrase Enrichment](https://mlsanity.com/paper/2407.01784/) (2024) - [Similarity-Dissimilarity Loss for Multi-label Supervised Contrastive Learning](https://mlsanity.com/paper/2410.13439/) (2024) - [Improving Arabic Multi-Label Emotion Classification using Stacked Embeddings and Hybrid Loss Function](https://mlsanity.com/paper/2410.03979/) (2024) - [Schema-Adaptive Tabular Representation Learning with LLMs for Generalizable Multimodal Clinical Reasoning](https://mlsanity.com/paper/2604.11835/) (2026) - [A Debiased Nearest Neighbors Framework for Multi-Label Text Classification](https://mlsanity.com/paper/2408.03202/) (2024) - [Seeing Through the PRISM: Compound & Controllable Restoration of Scientific Images](https://mlsanity.com/paper/2603.14151/) (2026) - [D2ACE: Multi-Label Batch Selection Guided by Dual Dynamics and Adaptive Correlation Enhancement](https://mlsanity.com/paper/2605.09400/) (2026) ## Abstract (arXiv) The effectiveness of contrastive learning technology in natural language processing tasks is yet to be explored and analyzed. How to construct positive and negative samples correctly and reasonably is the core challenge of contrastive learning. It is even harder to discover contrastive objects in multi-label text classification tasks. There are very few contrastive losses proposed previously. In this paper, we investigate the problem from a different angle by proposing five novel contrastive losses for multi-label text classification tasks. These are Strict Contrastive Loss (SCL), Intra-label Contrastive Loss (ICL), Jaccard Similarity Contrastive Loss (JSCL), Jaccard Similarity Probability Contrastive Loss (JSPCL), and Stepwise Label Contrastive Loss (SLCL). We explore the effectiveness of contrastive learning for multi-label text classification tasks by the employment of these novel losses and provide a set of baseline models for deploying contrastive learning techniques on specific tasks. We further perform an interpretable analysis of our approach to show how different components of contrastive learning losses play their roles. The experimental results show that our proposed contrastive losses can bring improvement to multi-label text classification tasks. Our work also explores how contrastive learning should be adapted for multi-label text classification tasks.