mlsanityData to 7 Oct 2026

Federated Continual Learning via Knowledge Fusion: A Survey

arXiv 2312.16475 ↗First posted 27 Dec 2023cs.LG cs.AIcited by 53

Not on the map. Why?

Abstract (arXiv)

Data privacy and silos are nontrivial and greatly challenging in many real-world applications. Federated learning is a decentralized approach to training models across multiple local clients without the exchange of raw data from client devices to global servers. However, existing works focus on a static data environment and ignore continual learning from streaming data with incremental tasks. Federated Continual Learning (FCL) is an emerging paradigm to address model learning in both federated and continual learning environments. The key objective of FCL is to fuse heterogeneous knowledge from different clients and retain knowledge of previous tasks while learning on new ones. In this work, we delineate federated learning and continual learning first and then discuss their integration, i.e., FCL, and particular FCL via knowledge fusion. In summary, our motivations are four-fold: we (1) raise a fundamental problem called ''spatial-temporal catastrophic forgetting'' and evaluate its impact on the performance using a well-known method called federated averaging (FedAvg), (2) integrate most of the existing FCL methods into two generic frameworks, namely synchronous FCL and asynchronous FCL, (3) categorize a large number of methods according to the mechanism involved in knowledge fusion, and finally (4) showcase an outlook on the future work of FCL.

Key facts. Cited by 53 papers in the map (data through 7 Oct 2026). Builds on 92 of its 145 references that are in the map. First posted on arXiv 27 Dec 2023.

Builds on

92 of 145 references.

References · 92

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  4. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  5. 2017Squeeze-and-Excitation Networkscited by 4,763
  6. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  7. 2017Prototypical Networks for Few-shot Learningcited by 3,001
  8. 2016Learning without Forgettingcited by 2,465
  9. 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,313
  10. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,221
  11. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,531
  12. 2016Progressive Neural Networkscited by 1,332
38 more
  1. 2018Continual Lifelong Learning with Neural Networks: A Reviewcited by 1,313
  2. 2014How transferable are features in deep neural networks?cited by 1,216
  3. 2017Continual Learning with Deep Generative Replaycited by 1,152
  4. 2018Experience Replay for Continual Learningcited by 1,074
  5. 2015Effective Approaches to Attention-based Neural Machine Translationcited by 1,024
  6. 2019Deep Leakage from Gradientscited by 1,004
  7. 2021Learning to Prompt for Continual Learningcited by 938
  8. 2017PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruningcited by 882
  9. 2019Towards Federated Learning at Scale: System Designcited by 826
  10. 2018How To Backdoor Federated Learningcited by 745
  11. 2020Inverting Gradients -- How easy is it to break privacy in federated learning?cited by 707
  12. 2018Overcoming catastrophic forgetting with hard attention to the taskcited by 672
  13. 2020The Future of Digital Health with Federated Learningcited by 625
  14. 2019Three scenarios for continual learningcited by 610
  15. 2019Federated Learning with Personalization Layerscited by 533
  16. 2018Progress & Compress: A scalable framework for continual learningcited by 516
  17. 2017Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Trainingcited by 471
  18. 2021Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Surveycited by 457
  19. 2019Robust and Communication-Efficient Federated Learning from Non-IID Datacited by 415
  20. 2018Adaptive Federated Learning in Resource Constrained Edge Computing Systemscited by 408
  21. 2021Data-Free Knowledge Distillation for Heterogeneous Federated Learningcited by 380
  22. 2019A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protectioncited by 371
  23. 2016Expert Gate: Lifelong Learning with a Network of Expertscited by 369
  24. 2020iDLG: Improved Deep Leakage from Gradientscited by 367
  25. 2017PathNet: Evolution Channels Gradient Descent in Super Neural Networkscited by 362
  26. 2017Measuring Catastrophic Forgetting in Neural Networkscited by 359
  27. 2014Recurrent Models of Visual Attentioncited by 344
  28. 2017Deep Models Under the GAN: Information Leakage from Collaborative Deep Learningcited by 326
  29. 2021Online Continual Learning in Image Classification: An Empirical Surveycited by 242
  30. 2018Knowledge Transfer via Distillation of Activation Boundaries Formed by Hidden Neuronscited by 235
  31. 2019BERT Post-Training for Review Reading Comprehension and Aspect-based Sentiment Analysiscited by 176
  32. 2020Threats to Federated Learning: A Surveycited by 171
  33. 2020Privacy and Robustness in Federated Learning: Attacks and Defensescited by 163
  34. 2020Federated Continual Learning with Weighted Inter-client Transfercited by 158
  35. 2020Survey of Personalization Techniques for Federated Learningcited by 144
  36. 2020Salvaging Federated Learning by Local Adaptationcited by 142
  37. 2021The Distributed Discrete Gaussian Mechanism for Federated Learning with Secure Aggregationcited by 127
  38. 2022Federated Class-Incremental Learningcited by 126

50 most cited of 92 All 92 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 13; 2025: 25; 2026: 15.

53 papers.

Sort:

By area:

  1. 2024Recent Advances of Multimodal Continual Learning: A Comprehensive Surveycited by 39
  2. 2024Diffusion-Driven Data Replay: A Novel Approach to Combat Forgetting in Federated Class Continual Learningcited by 23
  3. 2024Unleashing the Power of Continual Learning on Non-Centralized Devices: A Surveycited by 23
  4. 2025Ten Challenging Problems in Federated Foundation Modelscited by 20
  5. 2025A Survey of Continual Reinforcement Learningcited by 16
  6. 2024Rehearsal-Free Continual Federated Learning with Synergistic Synaptic Intelligencecited by 15
  7. 2025Unlearning through Knowledge Overwriting: Reversible Federated Unlearning via Selective Sparse Adaptercited by 15
  8. 2024Synergizing Foundation Models and Federated Learning: A Surveycited by 14
  9. 2025Federated Continual Learning: Concepts, Challenges, and Solutionscited by 13
  10. 2024Non-IID data in Federated Learning: A Survey with Taxonomy, Metrics, Methods, Frameworks and Future Directionscited by 12
  11. 2024Federated Continual Learning for Edge-AI: A Comprehensive Surveycited by 10
  12. 2025Federated Continual Instruction Tuningcited by 10
38 more
  1. 2024FedProK: Trustworthy Federated Class-Incremental Learning via Prototypical Feature Knowledge Transfercited by 9
  2. 2025Class-wise Balancing Data Replay for Federated Class-Incremental Learningcited by 9
  3. 2024Handling Spatial-Temporal Data Heterogeneity for Federated Continual Learning via Tail Anchorcited by 8
  4. 2025SacFL: Self-Adaptive Federated Continual Learning for Resource-Constrained End Devicescited by 8
  5. 2025Mitigating Catastrophic Forgetting with Adaptive Transformer Block Expansion in Federated Fine-Tuningcited by 7
  6. 2026Lifecycle-Aware Federated Continual Learning in Mobile Autonomous Systemscited by 7
  7. 2026UniSD: Towards a Unified Self-Distillation Framework for Large Language Modelscited by 6
  8. 2024On the Convergence of Continual Federated Learning Using Incrementally Aggregated Gradientscited by 5
  9. 2025Resource-Constrained Federated Continual Learning: What Does Matter?cited by 5
  10. 2025A Thorough Assessment of the Non-IID Data Impact in Federated Learningcited by 5
  11. 2024Overcoming Catastrophic Forgetting in Federated Class-Incremental Learning via Federated Global Twin Generatorcited by 4
  12. 2026Memory Bank Compression for Continual Adaptation of Large Language Modelscited by 4
  13. 2025AFCL: Analytic Federated Continual Learning for Spatio-Temporal Invariance of Non-IID Datacited by 3
  14. 2025ErrorEraser: Unlearning Data Bias for Improved Continual Learningcited by 3
  15. 2025ConSurv: Multimodal Continual Learning for Survival Analysiscited by 2
  16. 2024FedRewind: Rewinding Continual Model Exchange for Decentralized Federated Learningcited by 1
  17. 2025Orthogonal Calibration for Asynchronous Federated Learningcited by 1
  18. 2025FedSEA-LLaMA: A Secure, Efficient and Adaptive Federated Splitting Framework for Large Language Modelscited by 1
  19. 2025Enhancing Federated Class-Incremental Learning via Spatial-Temporal Statistics Aggregationcited by 1
  20. 2025GraphFedMIG: Tackling Class Imbalance in Federated Graph Learning via Mutual Information-Guided Generationcited by 1
  21. 2025Decentralized Dynamic Cooperation of Personalized Models for Federated Continual Learningcited by 1
  22. 2025Privacy-Aware Continual Self-Supervised Learning on Multi-Window Chest Computed Tomography for Domain-Shift Robustnesscited by 1
  23. 2026Federated continual learning: A comprehensive survey on lifelong and privacy-preserving learning over distributed and non-stationary datacited by 1
  24. 2024Federated Learning with New Knowledge: Fundamentals, Advances, and Futurescited by 0
  25. 2025Enhancing Trust Management System for Connected Autonomous Vehicles Using Machine Learning Methods: A Surveycited by 0
  26. 2025Avoid Forgetting by Preserving Global Knowledge Gradients in Federated Learning with Non-IID Datacited by 0
  27. 2025Shift Happens: Mixture of Experts based Continual Adaptation in Federated Learningcited by 0
  28. 2025Onboarding Without Forgetting: Hypernetwork Personalization with Data-Free Replay for Personalized Federated Learningcited by 0
  29. 2025Large-Small Model Collaborative Framework for Federated Continual Learningcited by 0
  30. 2025Enhanced Federated Deep Multi-View Clustering under Uncertainty Scenariocited by 0
  31. 2026Federated Continual Learning for Privacy-Preserving Hospital Imaging Classificationcited by 0
  32. 2026One-Shot Federated Clustering of Non-Independent Completely Distributed Datacited by 0
  33. 2026DriftGuard: Mitigating Asynchronous Data Drift in Federated Learningcited by 0
  34. 2026MLFCIL: A Multi-Level Forgetting Mitigation Framework for Federated Class-Incremental Learning in LEO Satellitescited by 0
  35. 2026From Selection to Scheduling: Federated Geometry-Aware Correction Makes Exemplar Replay Work Better under Continual Dynamic Heterogeneitycited by 0
  36. 2026PMF-CL: Pareto-Minimal-Forgetting Continual Learner for Conflicting Taskscited by 0
  37. 2026When Generator Replay Degrades: Projected Rehearsal Orchestration for Heterogeneous Federated Class-Incremental Learningcited by 0
  38. 2026Federated Learning for Global Carbon Emission Forecasting: A Hybrid Time-Series Approach with Statistical and Neural Modelscited by 0

50 most cited of 53 All 53 (JSON)