Contrastive Representations for Label Noise Require Fine-Tuning
Not on the map. Why?
Abstract (arXiv)
In this paper we show that the combination of a Contrastive representation with a label noise-robust classification head requires fine-tuning the representation in order to achieve state-of-the-art performances. Since fine-tuned representations are shown to outperform frozen ones, one can conclude that noise-robust classification heads are indeed able to promote meaningful representations if provided with a suitable starting point. Experiments are conducted to draw a comprehensive picture of performances by featuring six methods and nine noise instances of three different kinds (none, symmetric, and asymmetric). In presence of noise the experiments show that fine tuning of Contrastive representation allows the six methods to achieve better results than end-to-end learning and represent a new reference compare to the recent state of art. Results are also remarkable stable versus the noise level.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 31 of its 54 references that are in the map. First posted on arXiv 20 Aug 2021.
Builds on
31 of 54 references.
References · 31
- 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
- 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
- 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
- 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
- 2020Improved Baselines with Momentum Contrastive Learningcited by 2,282
- 2018Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labelscited by 1,053
- 2018Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labelscited by 994
- 2017A Closer Look at Memorization in Deep Networkscited by 936
- 2020DivideMix: Learning with Noisy Labels as Semi-supervised Learningcited by 691
- 2018Learning to Reweight Examples for Robust Deep Learningcited by 670
- 2016Making Deep Neural Networks Robust to Label Noise: a Loss Correction Approachcited by 663
- 2019Self-supervised Visual Feature Learning with Deep Neural Networks: A Surveycited by 656
19 more
- 2017MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labelscited by 652
- 2020Learning from Noisy Labels with Deep Neural Networks: A Surveycited by 580
- 2020A Survey on Contrastive Self-supervised Learningcited by 553
- 2019Symmetric Cross Entropy for Robust Learning with Noisy Labelscited by 499
- 2017Robust Loss Functions under Label Noise for Deep Neural Networkscited by 441
- 2018Using Trusted Data to Train Deep Networks on Labels Corrupted by Severe Noisecited by 216
- 2019Are Anchor Points Really Indispensable in Label-Noise Learning?cited by 208
- 2019Gradient Descent with Early Stopping is Provably Robust to Label Noise for Overparameterized Neural Networkscited by 158
- 2015Learning with Symmetric Label Noise: The Importance of Being Unhingedcited by 120
- 2019Meta Label Correction for Noisy Label Learningcited by 108
- 2020Rethinking Importance Weighting for Deep Learning under Distribution Shiftcited by 92
- 2021Contrast to Divide: Self-Supervised Pre-Training for Learning with Noisy Labelscited by 69
- 2020What Do Neural Networks Learn When Trained With Random Labels?cited by 58
- 2019On Symmetric Losses for Learning from Corrupted Labelscited by 53
- 2021Contrastive Learning Improves Model Robustness Under Label Noisecited by 36
- 2021Towards Understanding Deep Learning from Noisy Labels with Small-Loss Criterioncited by 36
- 2020Meta Transition Adaptation for Robust Deep Learning with Noisy Labelscited by 17
- 2019Robust Training with Ensemble Consensuscited by 11
- 2020Importance Reweighting for Biquality Learningcited by 4
Built on by
Not cited yet.