mlsanityData to 7 Oct 2026

Conditional Supervised Contrastive Learning for Fair Text Classification

arXiv 2205.11485 ↗First posted 23 May 2022cs.CL cs.AI cs.CYcited by 11

Not on the map. Why?

Abstract (arXiv)

Contrastive representation learning has gained much attention due to its superior performance in learning representations from both image and sequential data. However, the learned representations could potentially lead to performance disparities in downstream tasks, such as increased silencing of underrepresented groups in toxicity comment classification. In light of this challenge, in this work, we study learning fair representations that satisfy a notion of fairness known as equalized odds for text classification via contrastive learning. Specifically, we first theoretically analyze the connections between learning representations with a fairness constraint and conditional supervised contrastive objectives, and then propose to use conditional supervised contrastive objectives to learn fair representations for text classification. We conduct experiments on two text datasets to demonstrate the effectiveness of our approaches in balancing the trade-offs between task performance and bias mitigation among existing baselines for text classification. Furthermore, we also show that the proposed methods are stable in different hyperparameter settings.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 54 of its 73 references that are in the map. First posted on arXiv 23 May 2022.

Builds on

54 of 73 references.

References · 54

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  3. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  4. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  5. 2020Supervised Contrastive Learningcited by 2,803
  6. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,168
  7. 2016Equality of Opportunity in Supervised Learningcited by 2,000
  8. 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,838
  9. 2020Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hyperspherecited by 1,393
  10. 2020WILDS: A Benchmark of in-the-Wild Distribution Shiftscited by 1,366
  11. 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,304
  12. 2021Ethical and social risks of harm from Language Modelscited by 957
38 more
  1. 2020Language (Technology) is Power: A Critical Survey of "Bias" in NLPcited by 951
  2. 2019EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Taskscited by 800
  3. 2019Data-Efficient Image Recognition with Contrastive Predictive Codingcited by 672
  4. 2018Gender Bias in Coreference Resolution: Evaluation and Debiasing Methodscited by 657
  5. 2016Inherent Trade-Offs in the Fair Determination of Risk Scorescited by 582
  6. 2019On Variational Bounds of Mutual Informationcited by 550
  7. 2020Null It Out: Guarding Protected Attributes by Iterative Nullspace Projectioncited by 442
  8. 2018Gender Bias in Coreference Resolutioncited by 438
  9. 2017Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraintscited by 420
  10. 2019On Measuring Social Biases in Sentence Encoderscited by 397
  11. 2019Nuanced Metrics for Measuring Unintended Bias with Real Data for Text Classificationcited by 396
  12. 2017Preventing Fairness Gerrymandering: Auditing and Learning for Subgroup Fairnesscited by 381
  13. 2018Learning Adversarially Fair and Transferable Representationscited by 353
  14. 2019Bias in Bios: A Case Study of Semantic Representation Bias in a High-Stakes Settingcited by 346
  15. 2019Mitigating Gender Bias in Natural Language Processing: Literature Reviewcited by 287
  16. 2020Supervised Contrastive Learning for Pre-trained Language Model Fine-tuningcited by 275
  17. 2019Evaluating Gender Bias in Machine Translationcited by 247
  18. 2020Measuring and Reducing Gendered Correlations in Pre-trained Modelscited by 222
  19. 2021Parametric Contrastive Learningcited by 217
  20. 2019Identifying and Reducing Gender Bias in Word-Level Language Modelscited by 214
  21. 2019Gender Bias in Contextualized Word Embeddingscited by 203
  22. 2018Contextual Augmentation: Data Augmentation by Words with Paradigmatic Relationscited by 201
  23. 2021Societal Biases in Language Generation: Progress and Challengescited by 201
  24. 2018An efficient framework for learning sentence representationscited by 197
  25. 2020Social Biases in NLP Models as Barriers for Persons with Disabilitiescited by 182
  26. 2018Adversarial Removal of Demographic Attributes from Text Datacited by 166
  27. 2019Inherent Tradeoffs in Learning Fair Representationscited by 144
  28. 2021Harms of Gender Exclusivity and Challenges in Non-Binary Representation in Language Technologiescited by 128
  29. 2021Scaling Deep Contrastive Learning Batch Size under Memory Limited Setupcited by 123
  30. 2019Toward Gender-Inclusive Coreference Resolutioncited by 84
  31. 2019Conditional Learning of Fair Representationscited by 78
  32. 2021FairFil: Contrastive Neural Debiasing Method for Pretrained Text Encoderscited by 75
  33. 2022Fair Contrastive Learning for Facial Attribute Classificationcited by 71
  34. 2021Stereotype and Skew: Quantifying Gender Bias in Pre-trained and Fine-tuned Language Modelscited by 57
  35. 2020Improving Disentangled Text Representation Learning with Information-Theoretic Guidancecited by 52
  36. 2021Diverse Adversaries for Mitigating Bias in Trainingcited by 48
  37. 2021Your fairness may vary: Pretrained language model fairness in toxic text classificationcited by 38
  38. 2021A Novel Estimator of Mutual Information for Learning to Disentangle Textual Representationscited by 37

50 most cited of 54 All 54 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 5; 2024: 4; 2025: 0; 2026: 1.

11 papers.

  1. 2023Factorized Contrastive Learning: Going Beyond Multi-view Redundancycited by 73
  2. 2022MABEL: Attenuating Gender Bias using Textual Entailment Datacited by 31
  3. 2023Prompt Tuning Pushes Farther, Contrastive Learning Pulls Closer: A Two-Stage Approach to Mitigate Social Biasescited by 21
  4. 2024A Contrastive Learning Approach to Mitigate Bias in Speech Modelscited by 9
  5. 2023Erasure of Unaligned Attributes from Neural Representationscited by 6
  6. 2024Learning Label Hierarchy with Supervised Contrastive Learningcited by 3
  7. 2023Pushing the Accuracy-Group Robustness Frontier with Introspective Self-playcited by 2
  8. 2023A Novel Information-Theoretic Objective to Disentangle Representations for Fair Classificationcited by 1
  9. 2024Foundations of Multisensory Artificial Intelligencecited by 0
  10. 2024HateDebias: On the Diversity and Variability of Hate Speech Debiasingcited by 0
  11. 2026Dynamic Adaptive Attention and Supervised Contrastive Learning: A Novel Hybrid Framework for Text Sentiment Classificationcited by 0