mlsanityData to 7 Oct 2026

Not All Negatives are Equal: Label-Aware Contrastive Loss for Fine-grained Text Classification

arXiv 2109.05427 ↗First posted 12 Sep 2021cs.CL cs.AIcited by 43

Not on the map. Why?

Abstract (arXiv)

Fine-grained classification involves dealing with datasets with larger number of classes with subtle differences between them. Guiding the model to focus on differentiating dimensions between these commonly confusable classes is key to improving performance on fine-grained tasks. In this work, we analyse the contrastive fine-tuning of pre-trained language models on two fine-grained text classification tasks, emotion classification and sentiment analysis. We adaptively embed class relationships into a contrastive objective function to help differently weigh the positives and negatives, and in particular, weighting closely confusable negatives more than less similar negative examples. We find that Label-aware Contrastive Loss outperforms previous contrastive methods, in the presence of larger number and/or more confusable classes, and helps models to produce output distributions that are more differentiated.

Key facts. Cited by 43 papers in the map (data through 7 Oct 2026). Builds on 40 of its 56 references that are in the map. First posted on arXiv 12 Sep 2021.

Builds on

40 of 56 references.

References · 40

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  4. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  5. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  6. 2020Supervised Contrastive Learningcited by 2,803
  7. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,168
  8. 2019Contrastive Multiview Codingcited by 1,297
  9. 2018Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labelscited by 1,053
  10. 2019Self-Supervised Learning of Pretext-Invariant Representationscited by 806
  11. 2019EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Taskscited by 800
  12. 2016Sequence-Level Knowledge Distillationcited by 761
28 more
  1. 2019Learning Representations by Maximizing Mutual Information Across Viewscited by 733
  2. 2018Learning to Reweight Examples for Robust Deep Learningcited by 670
  3. 2020Recipes for building an open-domain chatbotcited by 593
  4. 2020Learning from Noisy Labels with Deep Neural Networks: A Surveycited by 580
  5. 2019Patient Knowledge Distillation for BERT Model Compressioncited by 502
  6. 2020GoEmotions: A Dataset of Fine-Grained Emotionscited by 479
  7. 2020Keystroke Dynamics as Part of Lifeloggingcited by 334
  8. 2020Combating noisy labels by agreement: A joint training method with co-regularizationcited by 289
  9. 2020Supervised Contrastive Learning for Pre-trained Language Model Fine-tuningcited by 275
  10. 2020CERT: Contrastive Self-supervised Learning for Language Understandingcited by 169
  11. 2014Evaluation of Output Embeddings for Fine-Grained Image Classificationcited by 166
  12. 2017Active Bias: Training More Accurate Neural Networks by Emphasizing High Variance Samplescited by 147
  13. 2019Fine-Grained Analysis of Propaganda in News Articlescited by 130
  14. 2021COCO-LM: Correcting and Contrasting Text Sequences for Language Model Pretrainingcited by 126
  15. 2019Fine-grained Fact Verification with Kernel Graph Attention Networkcited by 105
  16. 2019Knowledge Distillation from Internal Representationscited by 87
  17. 2020Contrastive Distillation on Intermediate Representations for Language Model Compressioncited by 57
  18. 2020SKEP: Sentiment Knowledge Enhanced Pre-training for Sentiment Analysiscited by 54
  19. 2019Fine-grained Sentiment Classification using BERTcited by 38
  20. 2017WASSA-2017 Shared Task on Emotion Intensitycited by 37
  21. 2020SentiBERT: A Transferable Transformer-Based Architecture for Compositional Sentiment Semanticscited by 30
  22. 2020BERT-EMD: Many-to-Many Layer Mapping for BERT Compression with Earth Mover's Distancecited by 27
  23. 2019Deep Learning for Fine-Grained Image Analysis: A Surveycited by 18
  24. 2018Evolutionary Data Measures: Understanding the Difficulty of Text Classification Taskscited by 11
  25. 2021Using Knowledge-Embedded Attention to Augment Pre-trained Language Models for Fine-Grained Emotion Recognitioncited by 10
  26. 2019Learning a Matching Model with Co-teaching for Multi-turn Response Selection in Retrieval-based Dialogue Systemscited by 7
  27. 2017Multitask Learning for Fine-Grained Twitter Sentiment Analysiscited by 5
  28. 2021Co-matching: Combating Noisy Labels by Augmentation Anchoringcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 12; 2023: 12; 2024: 11; 2025: 7; 2026: 1.

43 papers.

Sort:

By area:

  1. 2023AnglE-optimized Text Embeddingscited by 98
  2. 2024Large Language Models Produce Responses Perceived to be Empathiccited by 36
  3. 2023Facilitating Contrastive Learning of Discourse Relational Senses by Exploiting the Hierarchy of Sense Relationscited by 22
  4. 2022A Survey of Methods for Addressing Class Imbalance in Deep-Learning Based Natural Language Processingcited by 20
  5. 2024Recent advances in text embedding: A Comprehensive Review of Top-Performing Methods on the MTEB Benchmarkcited by 19
  6. 2022CPL: Counterfactual Prompt Learning for Vision and Language Modelscited by 18
  7. 2025Contrastive Representation Regularization for Vision-Language-Action Modelscited by 16
  8. 2022Fine-grained Category Discovery under Coarse-grained supervision with Hierarchical Weighted Self-contrastive Learningcited by 13
  9. 2022A Multi-level Supervised Contrastive Learning Framework for Low-Resource Natural Language Inferencecited by 8
  10. 2022Exploiting Contrastive Learning and Numerical Evidence for Confusing Legal Judgment Predictioncited by 8
  11. 2023Label-Aware Hyperbolic Embeddings for Fine-grained Emotion Classificationcited by 8
  12. 2024Simple-Sampling and Hard-Mixup with Prototypes to Rebalance Contrastive Learning for Text Classificationcited by 8
31 more
  1. 2022Label Anchored Contrastive Learning for Language Understandingcited by 7
  2. 2023Subjective Crowd Disagreements for Subjective Data: Uncovering Meaningful CrowdOpinion with Population-level Learningcited by 7
  3. 2023Joint Multiple Intent Detection and Slot Filling with Supervised Contrastive Learning and Self-Distillationcited by 7
  4. 2022CL-XABSA: Contrastive Learning for Cross-lingual Aspect-based Sentiment Analysiscited by 6
  5. 2023Using Positive Matching Contrastive Loss with Facial Action Units to mitigate bias in Facial Expression Recognitioncited by 4
  6. 2023W-procer: Weighted Prototypical Contrastive Learning for Medical Few-Shot Named Entity Recognitioncited by 4
  7. 2023Controlled Text Generation with Hidden Representation Transformationscited by 4
  8. 2023Instances and Labels: Hierarchy-aware Joint Supervised Contrastive Learning for Hierarchical Multi-Label Text Classificationcited by 4
  9. 2023DNA: Denoised Neighborhood Aggregation for Fine-grained Category Discoverycited by 4
  10. 2023The Skipped Beat: A Study of Sociopragmatic Understanding in LLMs for 64 Languagescited by 3
  11. 2024Learning Label Hierarchy with Supervised Contrastive Learningcited by 3
  12. 2022Type-enriched Hierarchical Contrastive Strategy for Fine-Grained Entity Typingcited by 2
  13. 2022A Benchmark Study of Contrastive Learning for Arabic Social Meaningcited by 2
  14. 2025CoDiEmb: A Collaborative yet Distinct Framework for Unified Representation Learning in Information Retrieval and Semantic Textual Similaritycited by 2
  15. 2022Contrastive Learning of Sociopragmatic Meaning in Social Mediacited by 1
  16. 2022Dictionary-Assisted Supervised Contrastive Learningcited by 1
  17. 2024Constrained Multi-Layer Contrastive Learning for Implicit Discourse Relationship Recognitioncited by 1
  18. 2025Semantic Adapter for Universal Text Embeddings: Diagnosing and Mitigating Negation Blindness to Enhance Universalitycited by 1
  19. 2025Supervised Contrastive Learning for Ordinal Engagement Measurementcited by 1
  20. 2025Bridging the Gap Between Multimodal Foundation Models and World Modelscited by 1
  21. 2022A Chinese Spelling Check Framework Based on Reverse Contrastive Learningcited by 0
  22. 2023Contrastive Bootstrapping for Label Refinementcited by 0
  23. 2024Visual Analytics for Fine-grained Text Classification Models and Datasetscited by 0
  24. 2024SensoryT5: Infusing Sensorimotor Norms into T5 for Enhanced Fine-grained Emotion Classificationcited by 0
  25. 2024Automatic Alignment of Discourse Relations of Different Discourse Annotation Frameworkscited by 0
  26. 2024Improved Text Emotion Prediction Using Combined Valence and Arousal Ordinal Classificationcited by 0
  27. 2024Improving Node Representation by Boosting Target-Aware Contrastive Losscited by 0
  28. 2024Class-Aware Contrastive Optimization for Imbalanced Text Classificationcited by 0
  29. 2025Advancing Sentiment Analysis: A Novel LSTM Framework with Multi-head Attentioncited by 0
  30. 2025Domain Lexical Knowledge-based Word Embedding Learning for Text Classification under Small Datacited by 0
  31. 2026Contextual Discrepancy-Aware Contrastive Learning for Robust Medical Time Series Diagnosis in Small-Sample Scenarioscited by 0