mlsanityData to 7 Oct 2026

When does Privileged Information Explain Away Label Noise?

arXiv 2303.01806 ↗First posted 3 Mar 2023cs.LG cs.CVcited by 10

Not on the map. Why?

Abstract (arXiv)

Leveraging privileged information (PI), or features available during training but not at test time, has recently been shown to be an effective method for addressing label noise. However, the reasons for its effectiveness are not well understood. In this study, we investigate the role played by different properties of the PI in explaining away label noise. Through experiments on multiple datasets with real PI (CIFAR-N/H) and a new large-scale benchmark ImageNet-PI, we find that PI is most helpful when it allows networks to easily distinguish clean from noisy data, while enabling a learning shortcut to memorize the noisy examples. Interestingly, when PI becomes too predictive of the target label, PI methods often perform worse than their no-PI baselines. Based on these findings, we propose several enhancements to the state-of-the-art PI methods and demonstrate the potential of PI as a means of tackling label noise. Finally, we show how we can easily combine the resulting PI approaches with existing no-PI techniques designed to deal with label noise.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 34 of its 45 references that are in the map. First posted on arXiv 3 Mar 2023.

Builds on

34 of 45 references.

References · 34

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  3. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  4. 2020Shortcut Learning in Deep Neural Networkscited by 1,928
  5. 2016Understanding deep learning requires rethinking generalizationcited by 1,657
  6. 2018Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labelscited by 1,053
  7. 2020DivideMix: Learning with Noisy Labels as Semi-supervised Learningcited by 691
  8. 2018An Empirical Study of Example Forgetting during Deep Neural Network Learningcited by 669
  9. 2016Making Deep Neural Networks Robust to Label Noise: a Loss Correction Approachcited by 663
  10. 2020Learning from Noisy Labels with Deep Neural Networks: A Surveycited by 580
  11. 2020Underspecification Presents Challenges for Credibility in Modern Machine Learningcited by 488
  12. 2020Early-Learning Regularization Prevents Memorization of Noisy Labelscited by 414
22 more
  1. 2020Gradient Starvation: A Learning Proclivity in Neural Networkscited by 262
  2. 2021Learning with Noisy Labels Revisited: A Study Using Real-World Human Annotationscited by 253
  3. 2019Human uncertainty makes classification more robustcited by 231
  4. 2015Unifying distillation and privileged informationcited by 218
  5. 2017Deep Learning is Robust to Massive Label Noisecited by 164
  6. 2020Learning with Instance-Dependent Label Noise: A Sample Sieve Approachcited by 158
  7. 2021Deep Learning Through the Lens of Example Difficultycited by 139
  8. 2017Learning From Noisy Large-Scale Datasets With Minimal Supervisioncited by 115
  9. 2021Understanding and Improving Early Stopping for Learning with Noisy Labelscited by 112
  10. 2022Robust Training under Label Noise by Over-parameterizationcited by 93
  11. 2021Uncertainty Baselines: Benchmarks for Uncertainty & Robustness in Deep Learningcited by 81
  12. 2018Removing Hidden Confounding by Experimental Groundingcited by 64
  13. 2020What Do Neural Networks Learn When Trained With Random Labels?cited by 58
  14. 2021Causally motivated Shortcut Removal Using Auxiliary Labelscited by 55
  15. 2020Automatic Shortcut Removal for Self-Supervised Representation Learningcited by 43
  16. 2021Correlated Input-Dependent Label Noise in Large-Scale Image Classificationcited by 37
  17. 2018Deep Learning under Privileged Information Using Heteroscedastic Dropoutcited by 26
  18. 2019High-Dimensional Linear Regression via Implicit Regularizationcited by 25
  19. 2022Toward Understanding Privileged Features Distillation in Learning-to-Rankcited by 16
  20. 2022Transfer and Marginalize: Explaining Away Label Noise with Privileged Informationcited by 15
  21. 2022Adapting Self-Supervised Vision Transformers by Probing Attention-Conditioned Masking Consistencycited by 11
  22. 2017MIML-FCN+: Multi-instance Multi-label Learning via Fully Convolutional Networks with Privileged Informationcited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 4; 2025: 3; 2026: 1.

10 papers.

  1. 2023URL: A Representation Learning Benchmark for Transferable Uncertainty Estimatescited by 17
  2. 2024Knockout: A simple way to handle missing inputscited by 4
  3. 2025Conformal Prediction with Corrupted Labels: Uncertain Imputation and Robust Re-weightingcited by 3
  4. 2023Pi-DUAL: Using Privileged Information to Distinguish Clean from Noisy Labelscited by 2
  5. 2024Rethinking Knowledge Transfer in Learning Using Privileged Informationcited by 2
  6. 2024Uncertainties of Latent Representations in Computer Visioncited by 1
  7. 2024Noisy Ostracods: A Fine-Grained, Imbalanced Real-World Dataset for Benchmarking Robust Machine Learning and Label Correction Methodscited by 1
  8. 2025Driving Style Recognition Like an Expert Using Semantic Privileged Information from Large Language Modelscited by 0
  9. 2025When are radiology reports useful for training medical image classifiers?cited by 0
  10. 2026TextClusterLab: An Integrated Framework for Reliable Text Clustering Studiescited by 0