# Contrastive Learning for Fair Representations

arXiv: https://arxiv.org/abs/2109.10645 · HTML: https://mlsanity.com/paper/2109.10645/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 39 references that are in the map.
- First posted on arXiv 22 Sep 2021.

## Builds on (most cited first, 29 of 29)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [Big Self-Supervised Models are Strong Semi-Supervised Learners](https://mlsanity.com/paper/2006.10029/) (2020)
- [Contrastive Multiview Coding](https://mlsanity.com/paper/1906.05849/) (2019)
- [Data-Efficient Image Recognition with Contrastive Predictive Coding](https://mlsanity.com/paper/1905.09272/) (2019)
- [Mitigating Unwanted Biases with Adversarial Learning](https://mlsanity.com/paper/1801.07593/) (2018)
- [Gender Bias in Coreference Resolution: Evaluation and Debiasing Methods](https://mlsanity.com/paper/1804.06876/) (2018)
- [Prototypical Contrastive Learning of Unsupervised Representations](https://mlsanity.com/paper/2005.04966/) (2020)
- [Null It Out: Guarding Protected Attributes by Iterative Nullspace Projection](https://mlsanity.com/paper/2004.07667/) (2020)
- [Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraints](https://mlsanity.com/paper/1707.09457/) (2017)
- [Bias in Bios: A Case Study of Semantic Representation Bias in a High-Stakes Setting](https://mlsanity.com/paper/1901.09451/) (2019)
- [Supervised Contrastive Learning for Pre-trained Language Model Fine-tuning](https://mlsanity.com/paper/2011.01403/) (2020)
- [Contrastive Clustering](https://mlsanity.com/paper/2009.09687/) (2020)
- [Demographic Dialectal Variation in Social Media: A Case Study of African-American English](https://mlsanity.com/paper/1608.08868/) (2016)
- [Adversarial Removal of Demographic Attributes from Text Data](https://mlsanity.com/paper/1808.06640/) (2018)
- [CERT: Contrastive Self-supervised Learning for Language Understanding](https://mlsanity.com/paper/2005.12766/) (2020)
- [Using millions of emoji occurrences to learn any-domain representations for detecting sentiment, emotion and sarcasm](https://mlsanity.com/paper/1708.00524/) (2017)
- [Towards Robust and Privacy-preserving Text Representations](https://mlsanity.com/paper/1805.06093/) (2018)
- [Diverse Adversaries for Mitigating Bias in Training](https://mlsanity.com/paper/2101.10001/) (2021)
- [Multilingual Twitter Corpus and Baselines for Evaluating Demographic Bias in Hate Speech Recognition](https://mlsanity.com/paper/2002.10361/) (2020)
- [Stereotypical Bias Removal for Hate Speech Detection Task using Knowledge-based Generalizations](https://mlsanity.com/paper/2001.05495/) (2020)
- [Privacy and Fairness in Recommender Systems via Adversarial Training of User Representations](https://mlsanity.com/paper/1807.03521/) (2018)
- [Understanding and Achieving Efficient Robustness with Adversarial Supervised Contrastive Learning](https://mlsanity.com/paper/2101.10027/) (2021)

## Built on by (most cited first, 30 of 34)

- [A Survey on Fairness in Large Language Models](https://mlsanity.com/paper/2308.10149/) (2023)
- [MABEL: Attenuating Gender Bias using Textual Entailment Data](https://mlsanity.com/paper/2210.14975/) (2022)
- [Learning from data with structured missingness](https://mlsanity.com/paper/2304.01429/) (2023)
- [Optimising Equal Opportunity Fairness in Model Training](https://mlsanity.com/paper/2205.02393/) (2022)
- [Debiasing Methods for Fairer Neural Models in Vision and Language Research: A Survey](https://mlsanity.com/paper/2211.05617/) (2022)
- [Prompt Tuning Pushes Farther, Contrastive Learning Pulls Closer: A Two-Stage Approach to Mitigate Social Biases](https://mlsanity.com/paper/2307.01595/) (2023)
- [Conditional Supervised Contrastive Learning for Fair Text Classification](https://mlsanity.com/paper/2205.11485/) (2022)
- [A Contrastive Learning Approach to Mitigate Bias in Speech Models](https://mlsanity.com/paper/2406.14686/) (2024)
- [M3Hop-CoT: Misogynous Meme Identification with Multimodal Multi-hop Chain-of-Thought](https://mlsanity.com/paper/2410.09220/) (2024)
- [Distributionally Generative Augmentation for Fair Facial Attribute Classification](https://mlsanity.com/paper/2403.06606/) (2024)
- [Erasure of Unaligned Attributes from Neural Representations](https://mlsanity.com/paper/2302.02997/) (2023)
- [fairlib: A Unified Framework for Assessing and Improving Classification Fairness](https://mlsanity.com/paper/2205.01876/) (2022)
- [Preserving Task-Relevant Information Under Linear Concept Removal](https://mlsanity.com/paper/2506.10703/) (2025)
- [Mitigating Confounding in Speech-Based Dementia Detection through Weight Masking](https://mlsanity.com/paper/2506.05610/) (2025)
- [Pushing the Accuracy-Group Robustness Frontier with Introspective Self-play](https://mlsanity.com/paper/2302.05807/) (2023)
- [Applying Intrinsic Debiasing on Downstream Tasks: Challenges and Considerations for Machine Translation](https://mlsanity.com/paper/2406.00787/) (2024)
- [Fairness via Independence: A (Conditional) Distance Covariance Framework](https://mlsanity.com/paper/2412.00720/) (2024)
- [Emotions are Subtle: Learning Sentiment Based Text Representations Using Contrastive Learning](https://mlsanity.com/paper/2112.01054/) (2021)
- [A Novel Information-Theoretic Objective to Disentangle Representations for Fair Classification](https://mlsanity.com/paper/2310.13990/) (2023)
- [LLM-Assisted Content Conditional Debiasing for Fair Text Embedding](https://mlsanity.com/paper/2402.14208/) (2024)
- [Collapsed Language Models Promote Fairness](https://mlsanity.com/paper/2410.04472/) (2024)
- [Debias-CLR: A Contrastive Learning Based Debiasing Method for Algorithmic Fairness in Healthcare Applications](https://mlsanity.com/paper/2411.10544/) (2024)
- [Achieving Fairness Without Harm via Selective Demographic Experts](https://mlsanity.com/paper/2511.06293/) (2025)
- [Benchmarking Bias Mitigation Toward Fairness Without Harm from Vision to LVLMs](https://mlsanity.com/paper/2602.03895/) (2026)
- [SHaSaM: Submodular Hard Sample Mining for Fair Facial Attribute Recognition](https://mlsanity.com/paper/2602.05162/) (2026)
- [Model and Evaluation: Towards Fairness in Multilingual Text Classification](https://mlsanity.com/paper/2303.15697/) (2023)
- [A Fair Classifier Embracing Triplet Collapse](https://mlsanity.com/paper/2306.04400/) (2023)
- [HateDebias: On the Diversity and Variability of Hate Speech Debiasing](https://mlsanity.com/paper/2406.04876/) (2024)
- [An Attention-based Framework for Fair Contrastive Learning](https://mlsanity.com/paper/2411.14765/) (2024)
- [A Survey on Stereotype Detection in Natural Language Processing](https://mlsanity.com/paper/2505.17642/) (2025)

## Abstract (arXiv)

Trained classification models can unintentionally lead to biased representations and predictions, which can reinforce societal preconceptions and stereotypes. Existing debiasing methods for classification models, such as adversarial training, are often expensive to train and difficult to optimise. In this paper, we propose a method for mitigating bias in classifier training by incorporating contrastive learning, in which instances sharing the same class label are encouraged to have similar representations, while instances sharing a protected attribute are forced further apart. In such a way our method learns representations which capture the task label in focused regions, while ensuring the protected attribute has diverse spread, and thus has limited impact on prediction and thereby results in fairer models. Extensive experimental results across four tasks in NLP and computer vision show (a) that our proposed method can achieve fairer representations and realises bias reductions compared with competitive baselines; and (b) that it can do so without sacrificing main task performance; (c) that it sets a new state-of-the-art performance in one task despite reducing the bias. Finally, our method is conceptually simple and agnostic to network architectures, and incurs minimal additional compute cost.