mlsanityData to 7 Oct 2026

Overcoming Forgetting in Federated Learning on Non-IID Data

arXiv 1910.07796 ↗First posted 17 Oct 2019cs.LG cs.CR stat.MLcited by 120

Not on the map. Why?

Abstract (arXiv)

We tackle the problem of Federated Learning in the non i.i.d. case, in which local models drift apart, inhibiting learning. Building on an analogy with Lifelong Learning, we adapt a solution for catastrophic forgetting to Federated Learning. We add a penalty term to the loss function, compelling all local models to converge to a shared optimum. We show that this can be done efficiently for communication (adding no further privacy risks), scaling with the number of nodes in the distributed setting. Our experiments show that this method is superior to competing ones for image recognition on the MNIST dataset.

Key facts. Cited by 120 papers in the map (data through 7 Oct 2026). First posted on arXiv 17 Oct 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 19; 2022: 22; 2023: 33; 2024: 29; 2025: 12; 2026: 5.

120 papers.

Sort:

By area:

  1. 2021Towards Personalized Federated Learningcited by 462
  2. 2021Federated Learning on Non-IID Data: A Surveycited by 345
  3. 2023Heterogeneous Federated Learning: State-of-the-art and Research Challengescited by 160
  4. 2022Federated Class-Incremental Learningcited by 126
  5. 2022Neurotoxin: Durable Backdoors in Federated Learningcited by 97
  6. 2023Federated Learning for Generalization, Robustness, Fairness: A Survey and Benchmarkcited by 97
  7. 2021Preservation of the Global Knowledge by Not-True Distillation in Federated Learningcited by 87
  8. 2023A Comprehensive Survey of Forgetting in Deep Learning Beyond Continual Learningcited by 66
  9. 2021Towards Fair Federated Learning with Zero-Shot Data Augmentationcited by 64
  10. 2023No One Left Behind: Real-World Federated Class-Incremental Learningcited by 53
  11. 2023Federated Continual Learning via Knowledge Fusion: A Surveycited by 53
  12. 2021FedCor: Correlation-Based Active Client Selection Strategy for Heterogeneous Federated Learningcited by 43
38 more
  1. 2022A Survey on Heterogeneous Federated Learningcited by 42
  2. 2021A Bayesian Federated Learning Framework with Online Laplace Approximationcited by 41
  3. 2023A Data-Free Approach to Mitigate Catastrophic Forgetting in Federated Class Incremental Learning for Vision Taskscited by 41
  4. 2023Learning Federated Visual Prompt in Null Space for MRI Reconstructioncited by 40
  5. 2023Generalizable Heterogeneous Federated Cross-Correlation and Instance Similarity Learningcited by 39
  6. 2023Cross-Silo Prototypical Calibration for Federated Learning with Non-IID Datacited by 35
  7. 2021Cross-Gradient Aggregation for Decentralized Learning from Non-IID datacited by 32
  8. 2022Knowledge Distillation for Federated Learning: a Practical Guidecited by 31
  9. 2022Acceleration of Federated Learning with Alleviated Forgetting in Local Trainingcited by 29
  10. 2023GradMA: A Gradient-Memory-based Accelerated Federated Learning with Alleviated Catastrophic Forgettingcited by 27
  11. 2024Taming Cross-Domain Representation Variance in Federated Prototype Learning with Heterogeneous Data Domainscited by 27
  12. 2023Privacy and Fairness in Federated Learning: on the Perspective of Trade-offcited by 26
  13. 2024FeDeRA:Efficient Fine-tuning of Language Models in Federated Learning Leveraging Weight Decompositioncited by 26
  14. 2023Federated Domain Generalization: A Surveycited by 25
  15. 2021Towards Federated Learning on Time-Evolving Heterogeneous Datacited by 23
  16. 2024Unleashing the Power of Continual Learning on Non-Centralized Devices: A Surveycited by 23
  17. 2022Federated Learning from Only Unlabeled Data with Class-Conditional-Sharing Clientscited by 20
  18. 2022SphereFed: Hyperspherical Federated Learningcited by 19
  19. 2021Adversarial training in communication constrained federated learningcited by 17
  20. 2023FedBone: Towards Large-Scale Federated Multi-Task Learningcited by 14
  21. 2021Privacy and Trust Redefined in Federated Machine Learningcited by 13
  22. 2021Federated mmWave Beam Selection Utilizing LIDAR Datacited by 12
  23. 2022Spatial-Temporal Federated Learning for Lifelong Person Re-identification on Distributed Edgescited by 12
  24. 2023A Survey on Class Imbalance in Federated Learningcited by 12
  25. 2023Fed-CPrompt: Contrastive Prompt for Rehearsal-Free Federated Continual Learningcited by 12
  26. 2024Advances in Robust Federated Learning: A Survey with Heterogeneity Considerationscited by 12
  27. 2024Green Federated Learning: A new era of Green Aware AIcited by 12
  28. 2022Federated Continual Learning for Text Classification via Selective Inter-client Transfercited by 9
  29. 2023Re-Weighted Softmax Cross-Entropy to Control Forgetting in Federated Learningcited by 9
  30. 2023FedYolo: Augmenting Federated Learning with Pretrained Transformerscited by 9
  31. 2024Continual Learning for Smart City: A Surveycited by 9
  32. 2025Federated Class-Incremental Learning: A Hybrid Approach Using Latent Exemplars and Data-Free Techniques to Address Local and Global Forgettingcited by 9
  33. 2023Experimenting with Normalization Layers in Federated Learning on non-IID scenarioscited by 8
  34. 2023Towards More Suitable Personalization in Federated Learning via Decentralized Partial Model Trainingcited by 8
  35. 2024One-Shot Sequential Federated Learning for Non-IID Data by Enhancing Local Model Diversitycited by 8
  36. 2024Knowledge Distillation in Federated Learning: a Survey on Long Lasting Challenges and New Solutionscited by 8
  37. 2024Fast-Convergent and Communication-Alleviated Heterogeneous Hierarchical Federated Learning in Autonomous Drivingcited by 8
  38. 2023Window-based Model Averaging Improves Generalization in Heterogeneous Federated Learningcited by 7

50 most cited of 120 All 120 (JSON)