mlsanityData to 7 Oct 2026

Transfer and Marginalize: Explaining Away Label Noise with Privileged Information

arXiv 2202.09244 ↗First posted 18 Feb 2022cs.LGcited by 15

Not on the map. Why?

Abstract (arXiv)

Supervised learning datasets often have privileged information, in the form of features which are available at training time but are not available at test time e.g. the ID of the annotator that provided the label. We argue that privileged information is useful for explaining away label noise, thereby reducing the harmful impact of noisy labels. We develop a simple and efficient method for supervised learning with neural networks: it transfers via weight sharing the knowledge learned with privileged information and approximately marginalizes over privileged information at test time. Our method, TRAM (TRansfer and Marginalize), has minimal training time overhead and has the same test-time cost as not using privileged information. TRAM performs strongly on CIFAR-10H, ImageNet and Civil Comments benchmarks.

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Builds on 21 of its 35 references that are in the map. First posted on arXiv 18 Feb 2022.

Builds on

21 of 35 references.

References · 21

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  4. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  5. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  6. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,827
  7. 2017What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?cited by 2,335
  8. 2020Learning from Noisy Labels with Deep Neural Networks: A Surveycited by 580
  9. 2018Born Again Neural Networkscited by 539
  10. 2015Variational Dropout and the Local Reparameterization Trickcited by 489
  11. 2018Universal Sentence Encodercited by 485
  12. 2019Human uncertainty makes classification more robustcited by 231
9 more
  1. 2015Unifying distillation and privileged informationcited by 218
  2. 2021Correlated Input-Dependent Label Noise in Large-Scale Image Classificationcited by 37
  3. 2018Deep Learning under Privileged Information Using Heteroscedastic Dropoutcited by 26
  4. 2020A Survey on Deep Learning with Noisy Labels: How to train your model when you cannot trust on the annotations?cited by 25
  5. 2019Privileged Features Distillation at Taobao Recommendationscited by 20
  6. 2013Learning Using Privileged Information: SVM+ and Weighted SVMcited by 13
  7. 2015Patterns for Learning with Side Informationcited by 11
  8. 2017MIML-FCN+: Multi-instance Multi-label Learning via Fully Convolutional Networks with Privileged Informationcited by 10
  9. 2014Mind the Nuisance: Gaussian Process Classification using Privileged Noisecited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 2; 2024: 4; 2025: 2; 2026: 5.

15 papers.

Sort:

By area:

  1. 2026Zooming without Zooming: Region-to-Image Distillation for Fine-Grained Multimodal Perceptioncited by 47
  2. 2022Eliciting and Learning with Soft Labels from Every Annotatorcited by 43
  3. 2024MathWriting: A Dataset For Handwritten Mathematical Expression Recognitioncited by 20
  4. 2024Experts Don't Cheat: Learning What You Don't Know By Predicting Pairscited by 18
  5. 2022Toward Understanding Privileged Features Distillation in Learning-to-Rankcited by 16
  6. 2024On Subjective Uncertainty Quantification and Calibration in Natural Language Generationcited by 14
  7. 2023When does Privileged Information Explain Away Label Noise?cited by 10
  8. 2025Conformal Prediction with Corrupted Labels: Uncertain Imputation and Robust Re-weightingcited by 3
  9. 2023Pi-DUAL: Using Privileged Information to Distinguish Clean from Noisy Labelscited by 2
  10. 2024Rethinking Knowledge Transfer in Learning Using Privileged Informationcited by 2
  11. 2025When are radiology reports useful for training medical image classifiers?cited by 0
  12. 2026Neurosymbolic Imitation Learning with Human Guidance: A Privileged Information Approachcited by 0
3 more
  1. 2026AIDE: Automated Instruction via Distilled Expertise for Reference-Free Motor Skill Coachingcited by 0
  2. 2026TRACE-GS: On-Policy Trajectory Distillation with Privileged Geometric Conditioning for Sparse-View 3DGS Restorationcited by 0
  3. 2026Quantifying the Value of Privileged Information Using a PAC-Bayesian Approachcited by 0