mlsanityData to 7 Oct 2026

Federated Continual Learning: Concepts, Challenges, and Solutions

arXiv 2502.07059 ↗First posted 10 Feb 2025cs.LG cs.AIcited by 13

Not on the map. Why?

Abstract (arXiv)

Federated Continual Learning (FCL) has emerged as a robust solution for collaborative model training in dynamic environments, where data samples are continuously generated and distributed across multiple devices. This survey provides a comprehensive review of FCL, focusing on key challenges such as heterogeneity, model stability, communication overhead, and privacy preservation. We explore various forms of heterogeneity and their impact on model performance. Solutions to non-IID data, resource-constrained platforms, and personalized learning are reviewed in an effort to show the complexities of handling heterogeneous data distributions. Next, we review techniques for ensuring model stability and avoiding catastrophic forgetting, which are critical in non-stationary environments. Privacy-preserving techniques are another aspect of FCL that have been reviewed in this work. This survey has integrated insights from federated learning and continual learning to present strategies for improving the efficacy and scalability of FCL systems, making it applicable to a wide range of real-world scenarios.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 122 of its 276 references that are in the map. First posted on arXiv 10 Feb 2025.

Builds on

122 of 276 references.

References · 122

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  4. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  5. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  6. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  7. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,827
  8. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,239
  9. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  10. 2014Unsupervised Domain Adaptation by Backpropagationcited by 1,906
  11. 2017Gradient Episodic Memory for Continual Learningcited by 1,871
  12. 2015Learning Transferable Features with Deep Adaptation Networkscited by 1,323
38 more
  1. 2017Adversarial Discriminative Domain Adaptationcited by 1,309
  2. 2019TinyBERT: Distilling BERT for Natural Language Understandingcited by 1,153
  3. 2017Continual Learning with Deep Generative Replaycited by 1,152
  4. 2014Convolutional Neural Networks for Sentence Classificationcited by 1,124
  5. 2018Federated Learning with Non-IID Datacited by 1,099
  6. 2021Learning to Prompt for Continual Learningcited by 938
  7. 2017PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruningcited by 882
  8. 2023A Comprehensive Survey of Continual Learning: Theory, Method and Applicationcited by 860
  9. 2018Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigencecited by 734
  10. 2018Overcoming catastrophic forgetting with hard attention to the taskcited by 672
  11. 2017Federated Multi-Task Learningcited by 669
  12. 2022DualPrompt: Complementary Prompting for Rehearsal-free Continual Learningcited by 635
  13. 2019Gradient based sample selection for online continual learningcited by 568
  14. 2021Federated Learning Based on Dynamic Regularizationcited by 555
  15. 2019Federated Learning with Differential Privacy: Algorithms and Performance Analysiscited by 528
  16. 2019Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraintscited by 499
  17. 2020MobileBERT: a Compact Task-Agnostic BERT for Resource-Limited Devicescited by 476
  18. 2020Learning under Concept Drift: A Reviewcited by 470
  19. 2021Towards Personalized Federated Learningcited by 462
  20. 2019Robust and Communication-Efficient Federated Learning from Non-IID Datacited by 415
  21. 2018Adaptive Federated Learning in Resource Constrained Edge Computing Systemscited by 408
  22. 2019Online Continual Learning with Maximally Interfered Retrievalcited by 376
  23. 2018Client Selection for Federated Learning with Heterogeneous Resources in Mobile Edgecited by 347
  24. 2020Personalized Federated Learning: A Meta-Learning Approachcited by 309
  25. 2019Improving Federated Learning Personalization via Model Agnostic Meta Learningcited by 264
  26. 2021Online Continual Learning in Image Classification: An Empirical Surveycited by 242
  27. 2020Federated learning with hierarchical clustering of local updates to improve training on non-IID datacited by 222
  28. 2018Multi-Adversarial Domain Adaptationcited by 195
  29. 2020Understanding the Role of Training Regimes in Continual Learningcited by 177
  30. 2017Image-Image Domain Adaptation with Preserved Self-Similarity and Domain-Dissimilarity for Person Re-identificationcited by 168
  31. 2020Federated Continual Learning with Weighted Inter-client Transfercited by 158
  32. 2024Continual Learning with Pre-Trained Models: A Surveycited by 151
  33. 2020Survey of Personalization Techniques for Federated Learningcited by 144
  34. 2019Federated Adversarial Domain Adaptationcited by 140
  35. 2020Adversarial Continual Learningcited by 140
  36. 2018Partial Adversarial Domain Adaptationcited by 127
  37. 2022Federated Class-Incremental Learningcited by 126
  38. 2018Don't forget, there is more than forgetting: new metrics for Continual Learningcited by 113

50 most cited of 122 All 122 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 10.

13 papers.

Sort:

By area:

  1. 2025Federated Learning for Large Models in Medical Imaging: A Comprehensive Reviewcited by 2
  2. 2025FedTeddi: Temporal Drift and Divergence Aware Scheduling for Timely Federated Edge Learningcited by 1
  3. 2026Federated continual learning: A comprehensive survey on lifelong and privacy-preserving learning over distributed and non-stationary datacited by 1
  4. 2026C2FL: Clustered Continual Federated Learning under Spatial and Temporal Driftcited by 1
  5. 2025Shift Happens: Mixture of Experts based Continual Adaptation in Federated Learningcited by 0
  6. 2026Catastrophic Forgetting Resilient One-Shot Incremental Federated Learningcited by 0
  7. 2026Agentic Federated Learning: The Future of Distributed Training Orchestrationcited by 0
  8. 2026Parameter Efficient Multi-Class Intelligent Scheduling for Multimodal Online Distributed Industrial Anomaly Detectioncited by 0
  9. 2026When Generator Replay Degrades: Projected Rehearsal Orchestration for Heterogeneous Federated Class-Incremental Learningcited by 0
  10. 2026SpeechLLM Meets Federated Learning for End-to-End ASR: English and Italian Case Studiescited by 0
  11. 2026Sylvas: Synergistic Learning Value based Device Scheduling in Federated Continual Learningcited by 0
  12. 2026MECAIL: Communication-Aware Incremental Learning for Object Detection with 14.6 KB Spatiotemporal Expertscited by 0
1 more
  1. 2026ReSCENE: Server-Side Replay for Structural Mitigation of Catastrophic Forgetting in Federated Continual Learningcited by 0