mlsanityData to 7 Oct 2026

A Framework using Contrastive Learning for Classification with Noisy Labels

arXiv 2104.09563 ↗First posted 19 Apr 2021cs.CV cs.LGcited by 5

Not on the map. Why?

Abstract (arXiv)

We propose a framework using contrastive learning as a pre-training task to perform image classification in the presence of noisy labels. Recent strategies such as pseudo-labeling, sample selection with Gaussian Mixture models, weighted supervised contrastive learning have been combined into a fine-tuning phase following the pre-training. This paper provides an extensive empirical study showing that a preliminary contrastive learning step brings a significant gain in performance when using different loss functions: non-robust, robust, and early-learning regularized. Our experiments performed on standard benchmarks and real-world datasets demonstrate that: i) the contrastive pre-training increases the robustness of any loss function to noisy labels and ii) the additional fine-tuning phase can further improve accuracy but at the cost of additional complexity.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 43 of its 53 references that are in the map. First posted on arXiv 19 Apr 2021.

Builds on

43 of 53 references.

References · 43

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  4. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  5. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  6. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  7. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,845
  8. 2020Supervised Contrastive Learningcited by 2,803
  9. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,282
  10. 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,838
  11. 2019Similarity of Neural Network Representations Revisitedcited by 1,695
  12. 2016Understanding deep learning requires rethinking generalizationcited by 1,657
31 more
  1. 2019MixMatch: A Holistic Approach to Semi-Supervised Learningcited by 1,390
  2. 2018Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labelscited by 1,053
  3. 2018Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labelscited by 994
  4. 2017A Closer Look at Memorization in Deep Networkscited by 936
  5. 2019Self-Supervised Learning of Pretext-Invariant Representationscited by 806
  6. 2020DivideMix: Learning with Noisy Labels as Semi-supervised Learningcited by 691
  7. 2019Data-Efficient Image Recognition with Contrastive Predictive Codingcited by 672
  8. 2016Making Deep Neural Networks Robust to Label Noise: a Loss Correction Approachcited by 663
  9. 2017MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labelscited by 652
  10. 2018Exploring the Limits of Weakly Supervised Pretrainingcited by 604
  11. 2020Learning from Noisy Labels with Deep Neural Networks: A Surveycited by 580
  12. 2019Symmetric Cross Entropy for Robust Learning with Noisy Labelscited by 499
  13. 2017Robust Loss Functions under Label Noise for Deep Neural Networkscited by 441
  14. 2020Early-Learning Regularization Prevents Memorization of Noisy Labelscited by 414
  15. 2020Hard Negative Mixing for Contrastive Learningcited by 399
  16. 2019How does Disagreement Help Generalization against Label Corruption?cited by 390
  17. 2020Keystroke Dynamics as Part of Lifeloggingcited by 334
  18. 2019Unsupervised Label Noise Modeling and Loss Correctioncited by 327
  19. 2020Combating noisy labels by agreement: A joint training method with co-regularizationcited by 289
  20. 2017WebVision Database: Visual Learning and Understanding from Web Datacited by 262
  21. 2020Normalized Loss Functions for Deep Learning with Noisy Labelscited by 257
  22. 2018Using Trusted Data to Train Deep Networks on Labels Corrupted by Severe Noisecited by 216
  23. 2019Are Anchor Points Really Indispensable in Label-Noise Learning?cited by 208
  24. 2019SELF: Learning to Filter Noisy Labels with Self-Ensemblingcited by 164
  25. 2020Multi-Objective Interpolation Training for Robustness to Label Noisecited by 86
  26. 2018Benchmark Analysis of Representative Deep Neural Network Architecturescited by 77
  27. 2020MoPro: Webly Supervised Learning with Momentum Prototypescited by 66
  28. 2020A Framework For Contrastive Self-Supervised Learning And Designing A New Approachcited by 51
  29. 2020SemiNLL: A Framework of Noisy-Label Learning by Semi-Supervised Learningcited by 11
  30. 2020No Regret Sample Selection with Noisy Labelscited by 4
  31. 2020Identifying Training Stop Point with Noisy Labeled Datacited by 2

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 2; 2023: 1; 2024: 0; 2025: 1.

5 papers.

  1. 2022On Learning Contrastive Representations for Learning with Noisy Labelscited by 21
  2. 2023SSL-Cleanse: Trojan Detection and Mitigation in Self-Supervised Learningcited by 11
  3. 2021From Face to Gait: Weakly-Supervised Learning of Gender Information from Walking Patternscited by 6
  4. 2025Multi-level Supervised Contrastive Learningcited by 1
  5. 2022Dropout can Simulate Exponential Number of Models for Sample Selection Techniquescited by 0