mlsanityData to 7 Oct 2026

Training Neural Networks on Data Sources with Unknown Reliability

arXiv 2212.02895 ↗First posted 6 Dec 2022cs.LG stat.MLcited by 0

Not on the map. Why?

Abstract (arXiv)

When data is generated by multiple sources, conventional training methods update models assuming equal reliability for each source and do not consider their individual data quality. However, in many applications, sources have varied levels of reliability that can have negative effects on the performance of a neural network. A key issue is that often the quality of the data for individual sources is not known during training. Previous methods for training models in the presence of noisy data do not make use of the additional information that the source label can provide. Focusing on supervised learning, we aim to train neural networks on each data source for a number of steps proportional to the source's estimated reliability by using a dynamic re-weighting strategy motivated by likelihood tempering. This way, we allow training on all sources during the warm-up and reduce learning on less reliable sources during the final training stages, when it has been shown that models overfit to noise. We show through diverse experiments that this can significantly improve model performance when trained on mixtures of reliable and unreliable data sources, and maintain performance when models are trained on reliable sources only.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 23 of its 56 references that are in the map. First posted on arXiv 6 Dec 2022.

Builds on

23 of 56 references.

References · 23

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  4. 2019A Survey on Bias and Fairness in Machine Learningcited by 1,911
  5. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,531
  6. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,461
  7. 2018Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labelscited by 1,053
  8. 2017A Closer Look at Memorization in Deep Networkscited by 936
  9. 2021Generalizing to Unseen Domains: A Survey on Domain Generalizationcited by 762
  10. 2017Exploring Generalization in Deep Learningcited by 595
  11. 2020Learning from Noisy Labels with Deep Neural Networks: A Surveycited by 580
  12. 2020GoEmotions: A Dataset of Fine-Grained Emotionscited by 479
11 more
  1. 2020Early-Learning Regularization Prevents Memorization of Noisy Labelscited by 414
  2. 2019How does Disagreement Help Generalization against Label Corruption?cited by 390
  3. 2014Stochastic Gradient Hamiltonian Monte Carlocited by 341
  4. 2019Unsupervised Label Noise Modeling and Loss Correctioncited by 327
  5. 2021Learning with Noisy Labels Revisited: A Study Using Real-World Human Annotationscited by 253
  6. 2020A Survey of Label-noise Representation Learning: Past, Present and Futurecited by 121
  7. 2018Iterative Learning with Open-set Noisy Labelscited by 108
  8. 2021To Smooth or Not? When Label Smoothing Meets Noisy Labelscited by 70
  9. 2022On Uncertainty, Tempering, and Data Augmentation in Bayesian Classificationcited by 46
  10. 2021Tackling Instance-Dependent Label Noise via a Universal Probabilistic Modelcited by 12
  11. 2021Auto-weighted Robust Federated Learning with Corrupted Data Sourcescited by 12

Built on by

Not cited yet.