# Unbiased Supervised Contrastive Learning

arXiv: https://arxiv.org/abs/2211.05568 · HTML: https://mlsanity.com/paper/2211.05568/

## Key facts

- Cited by 40 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 55 references that are in the map.
- First posted on arXiv 10 Nov 2022.

## Builds on (most cited first, 30 of 35)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness](https://mlsanity.com/paper/1811.12231/) (2018)
- [Natural Adversarial Examples](https://mlsanity.com/paper/1907.07174/) (2019)
- [Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels](https://mlsanity.com/paper/1805.07836/) (2018)
- [Adversarial Examples Are Not Bugs, They Are Features](https://mlsanity.com/paper/1905.02175/) (2019)
- [In Defense of the Triplet Loss for Person Re-Identification](https://mlsanity.com/paper/1703.07737/) (2017)
- [On Variational Bounds of Mutual Information](https://mlsanity.com/paper/1905.06922/) (2019)
- [Invariant Information Clustering for Unsupervised Image Classification and Segmentation](https://mlsanity.com/paper/1807.06653/) (2018)
- [Don't Take the Easy Way Out: Ensemble Based Methods for Avoiding Known Dataset Biases](https://mlsanity.com/paper/1909.03683/) (2019)
- [Approximating CNNs with Bag-of-local-Features models works surprisingly well on ImageNet](https://mlsanity.com/paper/1904.00760/) (2019)
- [Learning Not to Learn: Training Deep Neural Networks with Biased Data](https://mlsanity.com/paper/1812.10352/) (2018)
- [Towards Fairness in Visual Recognition: Effective Strategies for Bias Mitigation](https://mlsanity.com/paper/1911.11834/) (2019)
- [Learning De-biased Representations with Biased Representations](https://mlsanity.com/paper/1910.02806/) (2019)
- [Learning Fine-grained Image Similarity with Deep Ranking](https://mlsanity.com/paper/1404.4661/) (2014)
- [RUBi: Reducing Unimodal Biases in Visual Question Answering](https://mlsanity.com/paper/1906.10169/) (2019)
- [Machine Learning Models that Remember Too Much](https://mlsanity.com/paper/1709.07886/) (2017)
- [FairGAN: Fairness-aware Generative Adversarial Networks](https://mlsanity.com/paper/1805.11202/) (2018)
- [Learning from Failure: Training Debiased Classifier from Biased Classifier](https://mlsanity.com/paper/2007.02561/) (2020)
- [Controllable Invariance through Adversarial Feature Learning](https://mlsanity.com/paper/1705.11122/) (2017)
- [Learning Robust Representations by Projecting Superficial Statistics Out](https://mlsanity.com/paper/1903.06256/) (2019)
- [Learning Debiased Representation via Disentangled Feature Augmentation](https://mlsanity.com/paper/2107.01372/) (2021)
- [Turning a Blind Eye: Explicit Removal of Biases and Variation from Deep Neural Network Embeddings](https://mlsanity.com/paper/1809.02169/) (2018)
- [Large Margin Deep Networks for Classification](https://mlsanity.com/paper/1803.05598/) (2018)
- [EnD: Entangling and Disentangling deep representations for bias correction](https://mlsanity.com/paper/2103.02023/) (2021)
- [A Deeper Look at Dataset Bias](https://mlsanity.com/paper/1505.01257/) (2015)

## Built on by (most cited first, 30 of 40)

- [FLAC: Fairness-Aware Representation Learning by Suppressing Attribute-Class Associations](https://mlsanity.com/paper/2304.14252/) (2023)
- [Contrastive learning for regression in multi-site brain age prediction](https://mlsanity.com/paper/2211.08326/) (2022)
- [InfoCL: Alleviating Catastrophic Forgetting in Continual Text Classification from An Information Theoretic Perspective](https://mlsanity.com/paper/2310.06362/) (2023)
- [Double InfoGAN for Contrastive Analysis](https://mlsanity.com/paper/2401.17776/) (2024)
- [Separating common from salient patterns with Contrastive Representation Learning](https://mlsanity.com/paper/2402.11928/) (2024)
- [Mining bias-target Alignment from Voronoi Cells](https://mlsanity.com/paper/2305.03691/) (2023)
- [Revisiting Recommendation Loss Functions through Contrastive Learning (Technical Report)](https://mlsanity.com/paper/2312.08520/) (2023)
- [BAdd: Bias Mitigation through Bias Addition](https://mlsanity.com/paper/2408.11439/) (2024)
- [MAVias: Mitigate any Visual Bias](https://mlsanity.com/paper/2412.06632/) (2024)
- [Say My Name: a Model's Bias Discovery Framework](https://mlsanity.com/paper/2408.09570/) (2024)
- [SINCERE: Supervised Information Noise-Contrastive Estimation REvisited](https://mlsanity.com/paper/2309.14277/) (2023)
- [WISER: Weak supervISion and supErvised Representation learning to improve drug response prediction in cancer](https://mlsanity.com/paper/2405.04078/) (2024)
- [Diffusing DeBias: Synthetic Bias Amplification for Model Debiasing](https://mlsanity.com/paper/2502.09564/) (2025)
- [Processing and acquisition traces in visual encoders: What does CLIP know about your camera?](https://mlsanity.com/paper/2508.10637/) (2025)
- [Learning Common and Salient Generative Factors Between Two Image Datasets](https://mlsanity.com/paper/2512.12800/) (2025)
- [Debiasing surgeon: fantastic weights and how to find them](https://mlsanity.com/paper/2403.14200/) (2024)
- [NeuronTune: Towards Self-Guided Spurious Bias Mitigation](https://mlsanity.com/paper/2505.24048/) (2025)
- [Probabilistic Variational Contrastive Learning](https://mlsanity.com/paper/2506.10159/) (2025)
- [VB-Mitigator: An Open-source Framework for Evaluating and Advancing Visual Bias Mitigation](https://mlsanity.com/paper/2507.18348/) (2025)
- [RepCL: Exploring Effective Representation for Continual Text Classification](https://mlsanity.com/paper/2305.07289/) (2023)
- [Weakly-supervised positional contrastive learning: application to cirrhosis classification](https://mlsanity.com/paper/2307.04617/) (2023)
- [Marginal Debiased Network for Fair Visual Recognition](https://mlsanity.com/paper/2401.02150/) (2024)
- [AI-Assisted Diagnosis for Covid-19 CXR Screening: From Data Collection to Clinical Validation](https://mlsanity.com/paper/2405.11598/) (2024)
- [Unsupervised contrastive analysis for anomaly detection in brain MRIs via conditional diffusion models](https://mlsanity.com/paper/2406.00772/) (2024)
- [CosFairNet:A Parameter-Space based Approach for Bias Free Learning](https://mlsanity.com/paper/2410.15094/) (2024)
- [Multi-level Supervised Contrastive Learning](https://mlsanity.com/paper/2502.02202/) (2025)
- [Generalizing Supervised Contrastive learning: A Projection Perspective](https://mlsanity.com/paper/2506.09810/) (2025)
- [How does Labeling Error Impact Contrastive Learning? A Perspective from Data Dimensionality Reduction](https://mlsanity.com/paper/2507.11161/) (2025)
- [Transformed Multi-view 3D Shape Features with Contrastive Learning](https://mlsanity.com/paper/2510.19955/) (2025)
- [Reducing Class-Wise Performance Disparity via Margin Regularization](https://mlsanity.com/paper/2602.00205/) (2026)

## Abstract (arXiv)

Many datasets are biased, namely they contain easy-to-learn features that are highly correlated with the target class only in the dataset but not in the true underlying distribution of the data. For this reason, learning unbiased models from biased data has become a very relevant research topic in the last years. In this work, we tackle the problem of learning representations that are robust to biases. We first present a margin-based theoretical framework that allows us to clarify why recent contrastive losses (InfoNCE, SupCon, etc.) can fail when dealing with biased data. Based on that, we derive a novel formulation of the supervised contrastive loss (epsilon-SupInfoNCE), providing more accurate control of the minimal distance between positive and negative samples. Furthermore, thanks to our theoretical framework, we also propose FairKL, a new debiasing regularization loss, that works well even with extremely biased data. We validate the proposed losses on standard vision datasets including CIFAR10, CIFAR100, and ImageNet, and we assess the debiasing capability of FairKL with epsilon-SupInfoNCE, reaching state-of-the-art performance on a number of biased datasets, including real instances of biases in the wild.