# An Inclusive Theoretical Framework of Robust Supervised Contrastive Loss against Label Noise

arXiv: https://arxiv.org/abs/2501.01130 · HTML: https://mlsanity.com/paper/2501.01130/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 49 references that are in the map.
- First posted on arXiv 2 Jan 2025.

## Builds on (most cited first, 30 of 37)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere](https://mlsanity.com/paper/2005.10242/) (2020)
- [Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labels](https://mlsanity.com/paper/1804.06872/) (2018)
- [Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels](https://mlsanity.com/paper/1805.07836/) (2018)
- [DivideMix: Learning with Noisy Labels as Semi-supervised Learning](https://mlsanity.com/paper/2002.07394/) (2020)
- [Making Deep Neural Networks Robust to Label Noise: a Loss Correction Approach](https://mlsanity.com/paper/1609.03683/) (2016)
- [A Theoretical Analysis of Contrastive Unsupervised Representation Learning](https://mlsanity.com/paper/1902.09229/) (2019)
- [Symmetric Cross Entropy for Robust Learning with Noisy Labels](https://mlsanity.com/paper/1908.06112/) (2019)
- [Regularizing Neural Networks by Penalizing Confident Output Distributions](https://mlsanity.com/paper/1701.06548/) (2017)
- [Robust Loss Functions under Label Noise for Deep Neural Networks](https://mlsanity.com/paper/1712.09482/) (2017)
- [How does Disagreement Help Generalization against Label Corruption?](https://mlsanity.com/paper/1901.04215/) (2019)
- [Normalized Loss Functions for Deep Learning with Noisy Labels](https://mlsanity.com/paper/2006.13554/) (2020)
- [Using Trusted Data to Train Deep Networks on Labels Corrupted by Severe Noise](https://mlsanity.com/paper/1802.05300/) (2018)
- [Does label smoothing mitigate label noise?](https://mlsanity.com/paper/2003.02819/) (2020)
- [Dimensionality-Driven Learning with Noisy Labels](https://mlsanity.com/paper/1806.02612/) (2018)
- [Selective-Supervised Contrastive Learning with Noisy Labels](https://mlsanity.com/paper/2203.04181/) (2022)
- [Iterative Learning with Open-set Noisy Labels](https://mlsanity.com/paper/1804.00092/) (2018)
- [Masking: A New Perspective of Noisy Supervision](https://mlsanity.com/paper/1805.08193/) (2018)
- [Multi-Objective Interpolation Training for Robustness to Label Noise](https://mlsanity.com/paper/2012.04462/) (2020)
- [UNICON: Combating Label Noise Through Uniform Selection and Contrastive Learning](https://mlsanity.com/paper/2203.14542/) (2022)
- [Deep Learning over Multi-field Categorical Data: A Case Study on User Response Prediction](https://mlsanity.com/paper/1601.02376/) (2016)
- [Leveraged Weighted Loss for Partial Label Learning](https://mlsanity.com/paper/2106.05731/) (2021)
- [Class-Aware Contrastive Semi-Supervised Learning](https://mlsanity.com/paper/2203.02261/) (2022)
- [Open-set Label Noise Can Improve Robustness Against Inherent Label Noise](https://mlsanity.com/paper/2106.10891/) (2021)
- [Deep Learning from Noisy Image Labels with Quality Embedding](https://mlsanity.com/paper/1711.00583/) (2017)
- [Investigating Why Contrastive Learning Benefits Robustness Against Label Noise](https://mlsanity.com/paper/2201.12498/) (2022)
- [Robust Contrastive Learning against Noisy Views](https://mlsanity.com/paper/2201.04309/) (2022)

## Built on by (most cited first, 2 of 2)

- [Generalizing Supervised Contrastive learning: A Projection Perspective](https://mlsanity.com/paper/2506.09810/) (2025)
- [Noise-robust Contrastive Learning for Critical Transition Detection in Dynamical Systems](https://mlsanity.com/paper/2512.12523/) (2025)

## Abstract (arXiv)

Learning from noisy labels is a critical challenge in machine learning, with vast implications for numerous real-world scenarios. While supervised contrastive learning has recently emerged as a powerful tool for navigating label noise, many existing solutions remain heuristic, often devoid of a systematic theoretical foundation for crafting robust supervised contrastive losses. To address the gap, in this paper, we propose a unified theoretical framework for robust losses under the pairwise contrastive paradigm. In particular, we for the first time derive a general robust condition for arbitrary contrastive losses, which serves as a criterion to verify the theoretical robustness of a supervised contrastive loss against label noise. The theory indicates that the popular InfoNCE loss is in fact non-robust, and accordingly inspires us to develop a robust version of InfoNCE, termed Symmetric InfoNCE (SymNCE). Moreover, we highlight that our theory is an inclusive framework that provides explanations to prior robust techniques such as nearest-neighbor (NN) sample selection and robust contrastive loss. Validation experiments on benchmark datasets demonstrate the superiority of SymNCE against label noise.