# Co-learning: Learning from Noisy Labels with Self-supervision

arXiv: https://arxiv.org/abs/2108.04063 · HTML: https://mlsanity.com/paper/2108.04063/

## Key facts

- Cited by 51 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 45 references that are in the map.
- First posted on arXiv 5 Aug 2021.

## Builds on (most cited first, 30 of 38)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Exploring Simple Siamese Representation Learning](https://mlsanity.com/paper/2011.10566/) (2020)
- [Unsupervised Learning of Visual Features by Contrasting Cluster Assignments](https://mlsanity.com/paper/2006.09882/) (2020)
- [Unsupervised Representation Learning by Predicting Image Rotations](https://mlsanity.com/paper/1803.07728/) (2018)
- [Context Encoders: Feature Learning by Inpainting](https://mlsanity.com/paper/1604.07379/) (2016)
- [MixMatch: A Holistic Approach to Semi-Supervised Learning](https://mlsanity.com/paper/1905.02249/) (2019)
- [Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles](https://mlsanity.com/paper/1603.09246/) (2016)
- [Colorful Image Colorization](https://mlsanity.com/paper/1603.08511/) (2016)
- [Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labels](https://mlsanity.com/paper/1804.06872/) (2018)
- [Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels](https://mlsanity.com/paper/1805.07836/) (2018)
- [DivideMix: Learning with Noisy Labels as Semi-supervised Learning](https://mlsanity.com/paper/2002.07394/) (2020)
- [Making Deep Neural Networks Robust to Label Noise: a Loss Correction Approach](https://mlsanity.com/paper/1609.03683/) (2016)
- [MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labels](https://mlsanity.com/paper/1712.05055/) (2017)
- [Exploring the Limits of Weakly Supervised Pretraining](https://mlsanity.com/paper/1805.00932/) (2018)
- [Symmetric Cross Entropy for Robust Learning with Noisy Labels](https://mlsanity.com/paper/1908.06112/) (2019)
- [How does Disagreement Help Generalization against Label Corruption?](https://mlsanity.com/paper/1901.04215/) (2019)
- [Unsupervised Label Noise Modeling and Loss Correction](https://mlsanity.com/paper/1904.11238/) (2019)
- [Training Deep Neural Networks on Noisy Labels with Bootstrapping](https://mlsanity.com/paper/1412.6596/) (2014)
- [Joint Optimization Framework for Learning with Noisy Labels](https://mlsanity.com/paper/1803.11364/) (2018)
- [Combating noisy labels by agreement: A joint training method with co-regularization](https://mlsanity.com/paper/2003.02752/) (2020)
- [Normalized Loss Functions for Deep Learning with Noisy Labels](https://mlsanity.com/paper/2006.13554/) (2020)
- [YFCC100M: The New Data in Multimedia Research](https://mlsanity.com/paper/1503.01817/) (2015)
- [Decoupling "when to update" from "how to update"](https://mlsanity.com/paper/1706.02613/) (2017)
- [Using Trusted Data to Train Deep Networks on Labels Corrupted by Severe Noise](https://mlsanity.com/paper/1802.05300/) (2018)
- [Understanding and Utilizing Deep Neural Networks Trained with Noisy Labels](https://mlsanity.com/paper/1905.05040/) (2019)
- [CleanNet: Transfer Learning for Scalable Image Classifier Training with Label Noise](https://mlsanity.com/paper/1711.07131/) (2017)

## Built on by (most cited first, 30 of 51)

- [SimGRACE: A Simple Framework for Graph Contrastive Learning without Data Augmentation](https://mlsanity.com/paper/2202.03104/) (2022)
- [Temporal Attention Unit: Towards Efficient Spatiotemporal Predictive Learning](https://mlsanity.com/paper/2206.12126/) (2022)
- [ProGCL: Rethinking Hard Negative Mining in Graph Contrastive Learning](https://mlsanity.com/paper/2110.02027/) (2021)
- [Peer Review as A Multi-Turn and Long-Context Dialogue with Role-Based Interactions](https://mlsanity.com/paper/2406.05688/) (2024)
- [CVT-SLR: Contrastive Visual-Textual Transformation for Sign Language Recognition with Variational Alignment](https://mlsanity.com/paper/2303.05725/) (2023)
- [INTENT: Invariance and Discrimination-aware Noise Mitigation for Robust Composed Image Retrieval](https://mlsanity.com/paper/2604.18051/) (2026)
- [Identifying Hard Noise in Long-Tailed Sample Distribution](https://mlsanity.com/paper/2207.13378/) (2022)
- [DealMVC: Dual Contrastive Calibration for Multi-view Clustering](https://mlsanity.com/paper/2308.09000/) (2023)
- [Self-supervised Learning for Electroencephalogram: A Systematic Survey](https://mlsanity.com/paper/2401.05446/) (2024)
- [Hyperspherical Consistency Regularization](https://mlsanity.com/paper/2206.00845/) (2022)
- [SemiReward: A General Reward Model for Semi-supervised Learning](https://mlsanity.com/paper/2310.03013/) (2023)
- [Retrieval Meets Reasoning: Even High-school Textbook Knowledge Benefits Multimodal Reasoning](https://mlsanity.com/paper/2405.20834/) (2024)
- [Labeling Chaos to Learning Harmony: Federated Learning with Noisy Labels](https://mlsanity.com/paper/2208.09378/) (2022)
- [Enhancing Human-like Multi-Modal Reasoning: A New Challenging Dataset and Comprehensive Framework](https://mlsanity.com/paper/2307.12626/) (2023)
- [ALEX: Towards Effective Graph Transfer Learning with Noisy Labels](https://mlsanity.com/paper/2309.14673/) (2023)
- [Ambiguity-Restrained Text-Video Representation Learning for Partially Relevant Video Retrieval](https://mlsanity.com/paper/2506.07471/) (2025)
- [Dual-view Curricular Optimal Transport for Cross-lingual Cross-modal Retrieval](https://mlsanity.com/paper/2309.05451/) (2023)
- [Cross-Gate MLP with Protein Complex Invariant Embedding is A One-Shot Antibody Designer](https://mlsanity.com/paper/2305.09480/) (2023)
- [RobGC: Towards Robust Graph Condensation](https://mlsanity.com/paper/2406.13200/) (2024)
- [Teaching Yourself: Graph Self-Distillation on Neighborhood for Node Classification](https://mlsanity.com/paper/2210.02097/) (2022)
- [Handling Label Noise via Instance-Level Difficulty Modeling and Dynamic Optimization](https://mlsanity.com/paper/2505.00812/) (2025)
- [An Analysis of Concept Bottleneck Models: Measuring, Understanding, and Mitigating the Impact of Noisy Annotations](https://mlsanity.com/paper/2505.16705/) (2025)
- [Enhancing Sample Selection Against Label Noise by Cutting Mislabeled Easy Examples](https://mlsanity.com/paper/2502.08227/) (2025)
- [ChronoSelect: Robust Learning with Noisy Labels via Dynamics Temporal Memory](https://mlsanity.com/paper/2507.18183/) (2025)
- [Tripartite: Tackle Noisy Labels by a More Precise Partition](https://mlsanity.com/paper/2202.09579/) (2022)
- [Why Deep Models Often cannot Beat Non-deep Counterparts on Molecular Property Prediction?](https://mlsanity.com/paper/2306.17702/) (2023)
- [D2SP: Dynamic Dual-Stage Purification Framework for Dual Noise Mitigation in Vision-based Affective Recognition](https://mlsanity.com/paper/2406.16473/) (2024)
- [Tackling Instance-Dependent Label Noise with Dynamic Distribution Calibration](https://mlsanity.com/paper/2210.05126/) (2022)
- [EEG-MACS: Manifold Attention and Confidence Stratification for EEG-based Cross-Center Brain Disease Diagnosis under Unreliable Annotations](https://mlsanity.com/paper/2405.00734/) (2024)
- [Classifying Long-tailed and Label-noise Data via Disentangling and Unlearning](https://mlsanity.com/paper/2503.11414/) (2025)

## Abstract (arXiv)

Noisy labels, resulting from mistakes in manual labeling or webly data collecting for supervised learning, can cause neural networks to overfit the misleading information and degrade the generalization performance. Self-supervised learning works in the absence of labels and thus eliminates the negative impact of noisy labels. Motivated by co-training with both supervised learning view and self-supervised learning view, we propose a simple yet effective method called Co-learning for learning with noisy labels. Co-learning performs supervised learning and self-supervised learning in a cooperative way. The constraints of intrinsic similarity with the self-supervised module and the structural similarity with the noisily-supervised module are imposed on a shared common feature encoder to regularize the network to maximize the agreement between the two constraints. Co-learning is compared with peer methods on corrupted data from benchmark datasets fairly, and extensive results are provided which demonstrate that Co-learning is superior to many state-of-the-art approaches.