mlsanityData to 7 Oct 2026

Better Generative Replay for Continual Federated Learning

arXiv 2302.13001 ↗First posted 25 Feb 2023cs.LG cs.AIcited by 53

Not on the map. Why?

Abstract (arXiv)

Federated learning is a technique that enables a centralized server to learn from distributed clients via communications without accessing the client local data. However, existing federated learning works mainly focus on a single task scenario with static data. In this paper, we introduce the problem of continual federated learning, where clients incrementally learn new tasks and history data cannot be stored due to certain reasons, such as limited storage and data retention policy. Generative replay based methods are effective for continual learning without storing history data, but adapting them for this setting is challenging. By analyzing the behaviors of clients during training, we find that the unstable training process caused by distributed training on non-IID data leads to a notable performance degradation. To address this problem, we propose our FedCIL model with two simple but effective solutions: model consolidation and consistency enforcement. Our experimental results on multiple benchmark datasets demonstrate that our method significantly outperforms baselines.

Key facts. Cited by 53 papers in the map (data through 7 Oct 2026). Builds on 30 of its 44 references that are in the map. First posted on arXiv 25 Feb 2023.

Builds on

30 of 44 references.

References · 30

  1. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  3. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  4. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  5. 2016Learning without Forgettingcited by 2,465
  6. 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,313
  7. 2017Continual Learning with Deep Generative Replaycited by 1,152
  8. 2018Federated Learning with Non-IID Datacited by 1,099
  9. 2017Memory Aware Synapses: Learning what (not) to forgetcited by 1,094
  10. 2018Experience Replay for Continual Learningcited by 1,074
  11. 2020Dark Experience for General Continual Learning: a Strong, Simple Baselinecited by 895
  12. 2018LEAF: A Benchmark for Federated Settingscited by 780
18 more
  1. 2017Lifelong Learning with Dynamically Expandable Networkscited by 689
  2. 2019Three scenarios for continual learningcited by 610
  3. 2016Conditional Image Synthesis With Auxiliary Classifier GANscited by 562
  4. 2021Data-Free Knowledge Distillation for Heterogeneous Federated Learningcited by 380
  5. 2021Federated Learning on Non-IID Data: A Surveycited by 345
  6. 2020Understanding the Role of Training Regimes in Continual Learningcited by 177
  7. 2020FedBE: Making Bayesian Model Ensemble Applicable to Federated Learningcited by 176
  8. 2020Federated Continual Learning with Weighted Inter-client Transfercited by 158
  9. 2021Towards Understanding Knowledge Distillationcited by 158
  10. 2020Adversarial Continual Learningcited by 140
  11. 2022Federated Class-Incremental Learningcited by 126
  12. 2017Stabilizing Training of Generative Adversarial Networks through Regularizationcited by 118
  13. 2020FedGAN: Federated Generative Adversarial Networks for Distributed Datacited by 89
  14. 2021Rebooting ACGAN: Auxiliary Classifier GANs with Stable Trainingcited by 57
  15. 2021A distillation-based approach integrating continual learning and federated learning for pervasive servicescited by 42
  16. 2020Improving GAN Training with Probability Ratio Clipping and Sample Reweightingcited by 18
  17. 2021Federated Reconnaissance: Efficient, Distributed, Class-Incremental Learningcited by 14
  18. 2021Training Federated GANs with Theoretical Guarantees: A Universal Aggregation Approachcited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 9; 2024: 20; 2025: 14; 2026: 10.

53 papers.

Sort:

By area:

  1. 2023A Comprehensive Survey of Forgetting in Deep Learning Beyond Continual Learningcited by 66
  2. 2023A Data-Free Approach to Mitigate Catastrophic Forgetting in Federated Class Incremental Learning for Vision Taskscited by 41
  3. 2023Federated Incremental Semantic Segmentationcited by 36
  4. 2024Towards Efficient Replay in Federated Incremental Learningcited by 33
  5. 2024Diffusion-Driven Data Replay: A Novel Approach to Combat Forgetting in Federated Class Continual Learningcited by 23
  6. 2024Unleashing the Power of Continual Learning on Non-Centralized Devices: A Surveycited by 23
  7. 2023Federated Orthogonal Training: Mitigating Global Catastrophic Forgetting in Continual Federated Learningcited by 21
  8. 2024Personalized Federated Domain-Incremental Learning based on Adaptive Knowledge Matchingcited by 21
  9. 2024PILoRA: Prototype Guided Incremental LoRA for Federated Class-Incremental Learningcited by 19
  10. 2024Rehearsal-Free Continual Federated Learning with Synergistic Synaptic Intelligencecited by 15
  11. 2025Federated Continual Learning: Concepts, Challenges, and Solutionscited by 13
  12. 2025Accurate Forgetting for Heterogeneous Federated Continual Learningcited by 11
38 more
  1. 2024Towards Continual Learning Desiderata via HSIC-Bottleneck Orthogonalization and Equiangular Embeddingcited by 9
  2. 2025Class-wise Balancing Data Replay for Federated Class-Incremental Learningcited by 9
  3. 2025SacFL: Self-Adaptive Federated Continual Learning for Resource-Constrained End Devicescited by 8
  4. 2023Continual Adaptation of Vision Transformers for Federated Learningcited by 6
  5. 2023Adapter-based Selective Knowledge Distillation for Federated Multi-domain Meeting Summarizationcited by 6
  6. 2024Continual Offline Reinforcement Learning via Diffusion-based Dual Generative Replaycited by 6
  7. 2024Solving Continual Offline RL through Selective Weights Activation on Aligned Spacescited by 6
  8. 2024Federated Continual Learning Goes Online: Uncertainty-Aware Memory Management for Vision Tasks and Beyondcited by 5
  9. 2025Resource-Constrained Federated Continual Learning: What Does Matter?cited by 5
  10. 2023Don't Memorize; Mimic The Past: Federated Class Incremental Learning Without Episodic Memorycited by 4
  11. 2023Federated Continual Novel Class Learningcited by 4
  12. 2024Overcoming Catastrophic Forgetting in Federated Class-Incremental Learning via Federated Global Twin Generatorcited by 4
  13. 2024Data-Free Federated Class Incremental Learning with Diffusion-Based Generative Memorycited by 3
  14. 2024Masked Autoencoders are Parameter-Efficient Federated Continual Learnerscited by 3
  15. 2024Reminding Multimodal Large Language Models of Object-aware Knowledge with Retrieved Tagscited by 2
  16. 2024PIP: Prototypes-Injected Prompt for Federated Class Incremental Learningcited by 2
  17. 2024FPPL: An Efficient and Non-IID Robust Federated Continual Learning Frameworkcited by 2
  18. 2024Using Diffusion Models as Generative Replay in Continual Federated Learning -- What will Happen?cited by 2
  19. 2024Exemplar-condensed Federated Class-incremental Learningcited by 2
  20. 2025C^2Prompt: Class-aware Client Knowledge Interaction for Federated Continual Learningcited by 2
  21. 2025Data Heterogeneity and Forgotten Labels in Split Federated Learningcited by 2
  22. 2023Concept Matching: Clustering-based Federated Continual Learningcited by 1
  23. 2024Online-Score-Aided Federated Learning for Resource-Constrained Wireless Clients with Continual Data Arrivalcited by 1
  24. 2025Enhancing Federated Class-Incremental Learning via Spatial-Temporal Statistics Aggregationcited by 1
  25. 2025Improving Generalization in Heterogeneous Federated Continual Learning via Spatio-Temporal Gradient Matching with Prototypical Coresetcited by 1
  26. 2025Task-Agnostic Federated Continual Learning via Replay-Free Gradient Projectioncited by 1
  27. 2025Decentralized Dynamic Cooperation of Personalized Models for Federated Continual Learningcited by 1
  28. 2025FedGTEA: Federated Class-Incremental Learning with Gaussian Task Embedding and Alignmentcited by 1
  29. 2024Federated Learning with New Knowledge: Fundamentals, Advances, and Futurescited by 0
  30. 2025Asynchronous Federated Continual Segmentation with Evolving Clients and Label Spacescited by 0
  31. 2025FlexFed: Mitigating Catastrophic Forgetting in Heterogeneous Federated Learning in Pervasive Computing Environmentscited by 0
  32. 2026Server-Proximal Aggregation for Federated Domain-Incremental Learning under Partial Participation: Task-Uniform Convergence and Backward Transfercited by 0
  33. 2026FeDMRA: Federated Incremental Learning with Dynamic Memory Replay Allocationcited by 0
  34. 2026From Selection to Scheduling: Federated Geometry-Aware Correction Makes Exemplar Replay Work Better under Continual Dynamic Heterogeneitycited by 0
  35. 2026Federated Nested Learning: Collaborative Training of Self-Referential Memories for Test-Time Adaptationcited by 0
  36. 2026Canonicalized Stable-List Replay for Private Federated Continual Learning over Language-Model Embeddingscited by 0
  37. 2026FlashbackCL: Mitigating Temporal Forgetting in Federated Learningcited by 0
  38. 2026Accurate and Resource-Efficient Federated Continual Learningcited by 0

50 most cited of 53 All 53 (JSON)