mlsanityData to 14 Oct 2026

Revisiting Early-Learning Regularization When Federated Learning Meets Noisy Labels

arXiv 2402.05353 ↗First posted 8 Feb 2024cs.LG cs.DCcited by 1

Not on the map. Why?

Abstract (arXiv)

In the evolving landscape of federated learning (FL), addressing label noise presents unique challenges due to the decentralized and diverse nature of data collection across clients. Traditional centralized learning approaches to mitigate label noise are constrained in FL by privacy concerns and the heterogeneity of client data. This paper revisits early-learning regularization, introducing an innovative strategy, Federated Label-mixture Regularization (FLR). FLR adeptly adapts to FL's complexities by generating new pseudo labels, blending local and global model predictions. This method not only enhances the accuracy of the global model in both i.i.d. and non-i.i.d. settings but also effectively counters the memorization of noisy labels. Demonstrating compatibility with existing label noise and FL techniques, FLR paves the way for improved generalization in FL environments fraught with label inaccuracies.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 36 of its 51 references that are in the map. First posted on arXiv 8 Feb 2024.

Builds on

36 of 51 references.

References · 36

  1. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  3. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,183
  4. 2017On Calibration of Modern Neural Networkscited by 4,960
  5. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  6. 2016Understanding deep learning requires rethinking generalizationcited by 1,674
  7. 2018Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labelscited by 1,059
  8. 2016Temporal Ensembling for Semi-Supervised Learningcited by 1,057
  9. 2018Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labelscited by 997
  10. 2017A Closer Look at Memorization in Deep Networkscited by 942
  11. 2020DivideMix: Learning with Noisy Labels as Semi-supervised Learningcited by 692
  12. 2016Making Deep Neural Networks Robust to Label Noise: a Loss Correction Approachcited by 665
24 more
  1. 2020Learning from Noisy Labels with Deep Neural Networks: A Surveycited by 582
  2. 2021Federated Learning Based on Dynamic Regularizationcited by 562
  3. 2019Pseudo-Labeling and Confirmation Bias in Deep Semi-Supervised Learningcited by 510
  4. 2017Regularizing Neural Networks by Penalizing Confident Output Distributionscited by 464
  5. 2020Early-Learning Regularization Prevents Memorization of Noisy Labelscited by 416
  6. 2014Training Deep Neural Networks on Noisy Labels with Bootstrappingcited by 322
  7. 2020Data Poisoning Attacks Against Federated Learning Systemscited by 275
  8. 2021Learning with Noisy Labels Revisited: A Study Using Real-World Human Annotationscited by 256
  9. 2018Dimensionality-Driven Learning with Noisy Labelscited by 173
  10. 2019Deep learning with noisy labels: exploring techniques and remedies in medical image analysiscited by 173
  11. 2019Gradient Descent with Early Stopping is Provably Robust to Label Noise for Overparameterized Neural Networkscited by 159
  12. 2021Understanding and Improving Early Stopping for Learning with Noisy Labelscited by 113
  13. 2021FINE Samples for Learning with Noisy Labelscited by 82
  14. 2020Error-Bounded Correction of Noisy Labelscited by 74
  15. 2021To Smooth or Not? When Label Smoothing Meets Noisy Labelscited by 70
  16. 2022FedCorr: Multi-Stage Federated Learning for Label Noise Correctioncited by 61
  17. 2021Instance-dependent Label-noise Learning under a Structural Causal Modelcited by 50
  18. 2017Deep Learning from Noisy Image Labels with Quality Embeddingcited by 46
  19. 2019Attack-Resistant Federated Learning with Residual-based Reweightingcited by 40
  20. 2020Robust Federated Learning with Noisy Labelscited by 40
  21. 2020FOCUS: Dealing with Label Quality Disparity in Federated Learningcited by 36
  22. 2022Towards Federated Learning against Noisy Labels via Local Self-Regularizationcited by 26
  23. 2021Robust Federated Learning with Attack-Adaptive Aggregationcited by 17
  24. 2020How Does a Neural Network's Architecture Impact Its Robustness to Noisy Labels?cited by 12

Built on by

Cited by year

Citing papers per year: 2024: 1.

1 paper.

  1. 2024Tackling Noisy Clients in Federated Learning with End-to-end Label Correctioncited by 12