mlsanityData to 7 Oct 2026

GradMA: A Gradient-Memory-based Accelerated Federated Learning with Alleviated Catastrophic Forgetting

arXiv 2302.14307 ↗First posted 28 Feb 2023cs.CV cs.LGcited by 27

Not on the map. Why?

Abstract (arXiv)

Federated Learning (FL) has emerged as a de facto machine learning area and received rapid increasing research interests from the community. However, catastrophic forgetting caused by data heterogeneity and partial participation poses distinctive challenges for FL, which are detrimental to the performance. To tackle the problems, we propose a new FL approach (namely GradMA), which takes inspiration from continual learning to simultaneously correct the server-side and worker-side update directions as well as take full advantage of server's rich computing and memory resources. Furthermore, we elaborate a memory reduction strategy to enable GradMA to accommodate FL with a large scale of workers. We then analyze convergence of GradMA theoretically under the smooth non-convex setting and show that its convergence rate achieves a linear speed up w.r.t the increasing number of sampled active workers. At last, our extensive experiments on various image classification tasks show that GradMA achieves significant performance gains in accuracy and communication efficiency compared to SOTA baselines.

Key facts. Cited by 27 papers in the map (data through 7 Oct 2026). Builds on 38 of its 48 references that are in the map. First posted on arXiv 28 Feb 2023.

Builds on

38 of 48 references.

References · 38

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  3. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  4. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  5. 2019Advances and Open Problems in Federated Learningcited by 3,021
  6. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  7. 2017Gradient Episodic Memory for Continual Learningcited by 1,871
  8. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,531
  9. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,220
  10. 2020Adaptive Federated Optimizationcited by 1,036
  11. 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 836
  12. 2021Model-Contrastive Federated Learningcited by 762
26 more
  1. 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 678
  2. 2021Federated Learning Based on Dynamic Regularizationcited by 555
  3. 2021Data-Free Knowledge Distillation for Heterogeneous Federated Learningcited by 380
  4. 2019Orthogonal Gradient Descent for Continual Learningcited by 361
  5. 2021Gradient Projection Memory for Continual Learningcited by 338
  6. 2019Learn to Grow: A Continual Structure Learning Framework for Overcoming Catastrophic Forgettingcited by 302
  7. 2019Asynchronous Federated Optimizationcited by 264
  8. 2021Federated Learning for Internet of Things: A Comprehensive Surveycited by 247
  9. 2021Achieving Linear Speedup with Partial Worker Participation in Non-IID Federated Learningcited by 192
  10. 2022Fine-tuning Global Model via Data-Free Knowledge Distillation for Non-IID Federated Learningcited by 172
  11. 2020Federated Continual Learning with Weighted Inter-client Transfercited by 158
  12. 2020Mime: Mimicking Centralized Stochastic Algorithms in Federated Learningcited by 152
  13. 2021Training Networks in Null Space of Feature Covariance for Continual Learningcited by 139
  14. 2019Scalable and Order-robust Continual Learning with Additive Parameter Decompositioncited by 125
  15. 2019Overcoming Forgetting in Federated Learning on Non-IID Datacited by 120
  16. 2021FedCM: Federated Learning with Client-level Momentumcited by 77
  17. 2021Fast Federated Learning in the Presence of Arbitrary Device Unavailabilitycited by 72
  18. 2021HarmoFL: Harmonizing Local and Global Drifts in Federated Learning on Heterogeneous Medical Imagescited by 62
  19. 2022Communication-Efficient Adaptive Federated Learningcited by 55
  20. 2021Anarchic Federated Learningcited by 42
  21. 2022Multi-Level Branched Regularization for Federated Learningcited by 34
  22. 2021Cross-Gradient Aggregation for Decentralized Learning from Non-IID datacited by 32
  23. 2021Local-Global Knowledge Distillation in Heterogeneous Federated Learning with Non-IID Datacited by 31
  24. 2022Acceleration of Federated Learning with Alleviated Forgetting in Local Trainingcited by 29
  25. 2021A Hybrid Variance-Reduced Method for Decentralized Stochastic Non-Convex Optimizationcited by 21
  26. 2021Coordinating Momenta for Cross-silo Federated Learningcited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 7; 2024: 11; 2025: 8; 2026: 1.

27 papers.

Sort:

  1. 2023Federated Learning for Generalization, Robustness, Fairness: A Survey and Benchmarkcited by 97
  2. 2023A Comprehensive Survey of Forgetting in Deep Learning Beyond Continual Learningcited by 66
  3. 2023Federated Continual Learning via Knowledge Fusion: A Surveycited by 53
  4. 2024Fair Federated Learning under Domain Skew with Local Consistency and Domain Diversitycited by 28
  5. 2023FwdLLM: Efficient FedLLM using Forward Gradientcited by 25
  6. 2023DFRD: Data-Free Robustness Distillation for Heterogeneous Federated Learningcited by 23
  7. 2023FedSOL: Stabilized Orthogonal Learning with Proximal Restrictions in Federated Learningcited by 14
  8. 2024FedProK: Trustworthy Federated Class-Incremental Learning via Prototypical Feature Knowledge Transfercited by 9
  9. 2024Federated Class-Incremental Learning with Hierarchical Generative Prototypescited by 6
  10. 2024Flashback: Understanding and Mitigating Forgetting in Federated Learningcited by 4
  11. 2024FedCFA: Alleviating Simpson's Paradox in Model Aggregation with Counterfactual Federated Learningcited by 4
  12. 2024SNFinLLM: Systematic and Nuanced Financial Domain Adaptation of Chinese Large Language Modelscited by 2
15 more
  1. 2024Privacy-Preserving Federated Learning with Consistency via Knowledge Distillation Using Conditional Generatorcited by 2
  2. 2024DFDG: Data-Free Dual-Generator Adversarial Distillation for One-Shot Federated Learningcited by 2
  3. 2025pMixFed: Efficient Personalized Federated Learning through Adaptive Layer-Wise Mixupcited by 2
  4. 2023Energy-Efficient and Real-Time Sensing for Federated Continual Learning via Sample-Driven Controlcited by 1
  5. 2024Federated Cross-Training Learners for Robust Generalization under Data Heterogeneitycited by 1
  6. 2024TPFL: A Trustworthy Personalized Federated Learning Framework via Subjective Logiccited by 1
  7. 2025Addressing Label Shift in Distributed Learning via Entropy Regularizationcited by 1
  8. 2025Personalized Recommendation Models in Federated Settings: A Surveycited by 1
  9. 2025Improving Generalization in Heterogeneous Federated Continual Learning via Spatio-Temporal Gradient Matching with Prototypical Coresetcited by 1
  10. 2024Federated Impression for Learning with Distributed Heterogeneous Datacited by 0
  11. 2025FlexFed: Mitigating Catastrophic Forgetting in Heterogeneous Federated Learning in Pervasive Computing Environmentscited by 0
  12. 2025Intrinsic Training Signals for Federated Learning Aggregationcited by 0
  13. 2025DOLFIN: Balancing Stability and Plasticity in Federated Continual Learningcited by 0
  14. 2025Towards Robust Knowledge Removal in Federated Learning with High Data Heterogeneitycited by 0
  15. 2026FedADB: Class Anchor-Driven Dual-Branch Federated Learning for Mitigating Forgettingcited by 0