mlsanityData to 14 Oct 2026

Two Wrongs Don't Make a Right: Combating Confirmation Bias in Learning with Label Noise

arXiv 2112.02960 ↗First posted 6 Dec 2021cs.LGcited by 18

Not on the map. Why?

Abstract (arXiv)

Noisy labels damage the performance of deep networks. For robust learning, a prominent two-stage pipeline alternates between eliminating possible incorrect labels and semi-supervised training. However, discarding part of noisy labels could result in a loss of information, especially when the corruption has a dependency on data, e.g., class-dependent or instance-dependent. Moreover, from the training dynamics of a representative two-stage method DivideMix, we identify the domination of confirmation bias: pseudo-labels fail to correct a considerable amount of noisy labels, and consequently, the errors accumulate. To sufficiently exploit information from noisy labels and mitigate wrong corrections, we propose Robust Label Refurbishment (Robust LR) a new hybrid method that integrates pseudo-labeling and confidence estimation techniques to refurbish noisy labels. We show that our method successfully alleviates the damage of both label noise and confirmation bias. As a result, it achieves state-of-the-art performance across datasets and noise types, namely CIFAR under different levels of synthetic noise and Mini-WebVision and ANIMAL-10N with real-world noise.

Key facts. Cited by 18 papers in the map (data through 14 Oct 2026). Builds on 44 of its 58 references that are in the map. First posted on arXiv 6 Dec 2021.

Builds on

44 of 58 references.

References · 44

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  4. 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,269
  5. 2020FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidencecited by 2,085
  6. 2016Understanding deep learning requires rethinking generalizationcited by 1,674
  7. 2017Improved Regularization of Convolutional Neural Networks with Cutoutcited by 1,628
  8. 2016Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learningcited by 1,512
  9. 2019MixMatch: A Holistic Approach to Semi-Supervised Learningcited by 1,391
  10. 2018Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labelscited by 1,059
  11. 2017A Closer Look at Memorization in Deep Networkscited by 942
  12. 2020DivideMix: Learning with Noisy Labels as Semi-supervised Learningcited by 692
32 more
  1. 2016Making Deep Neural Networks Robust to Label Noise: a Loss Correction Approachcited by 665
  2. 2017MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labelscited by 653
  3. 2020Learning from Noisy Labels with Deep Neural Networks: A Surveycited by 582
  4. 2019Pseudo-Labeling and Confirmation Bias in Deep Semi-Supervised Learningcited by 510
  5. 2017Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning resultscited by 472
  6. 2017Regularizing Neural Networks by Penalizing Confident Output Distributionscited by 464
  7. 2019How does Disagreement Help Generalization against Label Corruption?cited by 390
  8. 2019Unsupervised Label Noise Modeling and Loss Correctioncited by 328
  9. 2014Training Deep Neural Networks on Noisy Labels with Bootstrappingcited by 322
  10. 2018Joint Optimization Framework for Learning with Noisy Labelscited by 293
  11. 2017WebVision Database: Visual Learning and Understanding from Web Datacited by 262
  12. 2017Decoupling "when to update" from "how to update"cited by 230
  13. 2019Probabilistic End-to-end Noise Correction for Learning with Noisy Labelscited by 210
  14. 2019Understanding and Utilizing Deep Neural Networks Trained with Noisy Labelscited by 209
  15. 2018Dimensionality-Driven Learning with Noisy Labelscited by 173
  16. 2019SELF: Learning to Filter Noisy Labels with Self-Ensemblingcited by 164
  17. 2018Learning to Learn from Noisy Labeled Datacited by 160
  18. 2019NLNL: Negative Learning for Noisy Labelscited by 151
  19. 2017Active Bias: Training More Accurate Neural Networks by Emphasizing High Variance Samplescited by 147
  20. 2020Dual T: Reducing Estimation Error for Transition Matrix in Label-noise Learningcited by 142
  21. 2020Self-Adaptive Training: beyond Empirical Risk Minimizationcited by 126
  22. 2021Learning with Feature-Dependent Label Noise: A Progressive Approachcited by 102
  23. 2019Image Classification with Deep Learning in the Presence of Noisy Labels: A Surveycited by 101
  24. 2015Webly Supervised Learning of Convolutional Networkscited by 95
  25. 2017Learning with Biased Complementary Labelscited by 93
  26. 2017Learning from Complementary Labelscited by 88
  27. 2021Augmentation Strategies for Learning with Noisy Labelscited by 86
  28. 2020Beyond Class-Conditional Assumption: A Primary Attempt to Combat Instance-Dependent Label Noisecited by 82
  29. 2020A Topological Filter for Learning with Label Noisecited by 76
  30. 2020Confidence Scores Make Instance-dependent Label-noise Learning Possiblecited by 63
  31. 2021Boosting Co-teaching with Compression Regularization for Label Noisecited by 31
  32. 2018A Semi-Supervised Two-Stage Approach to Learning from Noisy Labelscited by 29

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 2; 2023: 4; 2024: 8; 2025: 2; 2026: 2.

18 papers.

Sort:

By area:

  1. 2023BiCro: Noisy Correspondence Rectification for Multi-modality Data via Bi-directional Cross-modal Similarity Consistencycited by 27
  2. 2024On the Noise Robustness of In-Context Learning for Text Generationcited by 16
  3. 2024Learning to Rematch Mismatched Pairs for Robust Cross-Modal Retrievalcited by 11
  4. 2024Noisy Label Processing for Classification: A Surveycited by 8
  5. 2022Dynamic Loss For Robust Learningcited by 6
  6. 2024Learning with Imbalanced Noisy Data by Preventing Bias in Sample Selectioncited by 6
  7. 2022SplitNet: Learnable Clean-Noisy Label Splitting for Learning with Noisy Labelscited by 5
  8. 2023Noisy Correspondence Learning with Self-Reinforcing Errors Mitigationcited by 5
  9. 2024D2SP: Dynamic Dual-Stage Purification Framework for Dual Noise Mitigation in Vision-based Affective Recognitioncited by 3
  10. 2025Benchmarking Real-World Medical Image Classification with Noisy Labels: Challenges, Practice, and Outlookcited by 3
  11. 2025TAROT: Towards Essentially Domain-Invariant Robustness with Theoretical Justificationcited by 2
  12. 2023LaplaceConfidence: a Graph-based Approach for Learning with Noisy Labelscited by 1
6 more
  1. 2024Training Gradient Boosted Decision Trees on Tabular Data Containing Label Noise for Classification Taskscited by 1
  2. 2023Hypothesis Testing for Class-Conditional Noise Using Local Maximum Likelihoodcited by 0
  3. 2024Addressing Long-Tail Noisy Label Learning Problems: a Two-Stage Solution with Label Refurbishment Considering Label Raritycited by 0
  4. 2024Jump-teaching: Combating Sample Selection Bias via Temporal Disagreementcited by 0
  5. 2026TecoPrompt: Temporal-Conservative Prompt Learning for Vision-Language Modelscited by 0
  6. 2026GERIS: A Game-Theoretic Framework for Filtering Instance-Dependent Label Noise in License Plate Data Augmentationcited by 0