mlsanityData to 13 Oct 2026

Towards Efficient Replay in Federated Incremental Learning

arXiv 2403.05890 ↗First posted 9 Mar 2024cs.LG cs.DCcited by 33

Not on the map. Why?

Abstract (arXiv)

In Federated Learning (FL), the data in each client is typically assumed fixed or static. However, data often comes in an incremental manner in real-world applications, where the data domain may increase dynamically. In this work, we study catastrophic forgetting with data heterogeneity in Federated Incremental Learning (FIL) scenarios where edge clients may lack enough storage space to retain full data. We propose to employ a simple, generic framework for FIL named Re-Fed, which can coordinate each client to cache important samples for replay. More specifically, when a new task arrives, each client first caches selected previous samples based on their global and local importance. Then, the client trains the local model with both the cached samples and the samples from the new task. Theoretically, we analyze the ability of Re-Fed to discover important samples for replay thus alleviating the catastrophic forgetting problem. Moreover, we empirically show that Re-Fed achieves competitive performance compared to state-of-the-art methods.

Key facts. Cited by 33 papers in the map (data through 13 Oct 2026). Builds on 31 of its 50 references that are in the map. First posted on arXiv 9 Mar 2024.

Builds on

31 of 50 references.

References · 31

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,950
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,653
  3. 2015Domain-Adversarial Training of Neural Networkscited by 3,030
  4. 2018Federated Optimization in Heterogeneous Networkscited by 2,981
  5. 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,323
  6. 2018Moment Matching for Multi-Source Domain Adaptationcited by 1,328
  7. 2015Learning Transferable Features with Deep Adaptation Networkscited by 1,324
  8. 2017An Overview of Multi-Task Learning in Deep Neural Networkscited by 799
  9. 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 682
  10. 2018An Empirical Study of Example Forgetting during Deep Neural Network Learningcited by 675
  11. 2019Three scenarios for continual learningcited by 612
  12. 2021Deep Learning on a Data Diet: Finding Important Examples Early in Trainingcited by 532
19 more
  1. 2021Data-Free Knowledge Distillation for Heterogeneous Federated Learningcited by 383
  2. 2017PathNet: Evolution Channels Gradient Descent in Super Neural Networkscited by 363
  3. 2018Communication-Efficient On-Device Machine Learning: Federated Distillation and Augmentation under Non-IID Private Datacited by 243
  4. 2020Semantic Drift Compensation for Class-Incremental Learningcited by 241
  5. 2018Re-evaluating Continual Learning Scenarios: A Categorization and Case for Strong Baselinescited by 228
  6. 2020Federated Learning of a Mixture of Global and Local Modelscited by 209
  7. 2021Federated Learning for Smart Healthcare: A Surveycited by 183
  8. 2022Fine-tuning Global Model via Data-Free Knowledge Distillation for Non-IID Federated Learningcited by 172
  9. 2020Federated Continual Learning with Weighted Inter-client Transfercited by 160
  10. 2022Federated Class-Incremental Learningcited by 127
  11. 2018Continuous Learning in Single-Incremental-Task Scenarioscited by 109
  12. 2020Generative Feature Replay For Class-Incremental Learningcited by 109
  13. 2020Continual Learning with Node-Importance based Adaptive Group Sparse Regularizationcited by 103
  14. 2022An Efficient Domain-Incremental Learning Approach to Drive in All Weather Conditionscited by 54
  15. 2023Better Generative Replay for Continual Federated Learningcited by 53
  16. 2021Non-IID data and Continual Learning processes in Federated Learning: A long road aheadcited by 36
  17. 2022Federated Learning with Position-Aware Neuronscited by 24
  18. 2021Domain-Incremental Continual Learning for Mitigating Bias in Facial Expression and Action Unit Recognitioncited by 19
  19. 2020Optimization and Generalization of Regularization-Based Continual Learning: a Loss Approximation Viewpointcited by 10

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2024: 5; 2025: 19; 2026: 9.

33 papers.

Sort:

By area:

  1. 2024Personalized Federated Domain-Incremental Learning based on Adaptive Knowledge Matchingcited by 21
  2. 2025Ten Challenging Problems in Federated Foundation Modelscited by 20
  3. 2024Rehearsal-Free Continual Federated Learning with Synergistic Synaptic Intelligencecited by 15
  4. 2025Federated Continual Learning: Concepts, Challenges, and Solutionscited by 13
  5. 2025Class-wise Balancing Data Replay for Federated Class-Incremental Learningcited by 9
  6. 2024Handling Spatial-Temporal Data Heterogeneity for Federated Continual Learning via Tail Anchorcited by 8
  7. 2025SacFL: Self-Adaptive Federated Continual Learning for Resource-Constrained End Devicescited by 8
  8. 2024On the Convergence of Continual Federated Learning Using Incrementally Aggregated Gradientscited by 5
  9. 2025Resource-Constrained Federated Continual Learning: What Does Matter?cited by 5
  10. 2025Feature Distillation is the Better Choice for Model-Heterogeneous Federated Learningcited by 5
  11. 2025C^2Prompt: Class-aware Client Knowledge Interaction for Federated Continual Learningcited by 2
  12. 2025Data Heterogeneity and Forgotten Labels in Split Federated Learningcited by 2
21 more
  1. 2024Better Knowledge Enhancement for Privacy-Preserving Cross-Project Defect Predictioncited by 1
  2. 2025Enhancing Federated Class-Incremental Learning via Spatial-Temporal Statistics Aggregationcited by 1
  3. 2025Improving Generalization in Heterogeneous Federated Continual Learning via Spatio-Temporal Gradient Matching with Prototypical Coresetcited by 1
  4. 2025FedTeddi: Temporal Drift and Divergence Aware Scheduling for Timely Federated Edge Learningcited by 1
  5. 2025FedBiF: Communication-Efficient Federated Learning via Bits Freezingcited by 1
  6. 2025Task-Agnostic Federated Continual Learning via Replay-Free Gradient Projectioncited by 1
  7. 2025FedGTEA: Federated Class-Incremental Learning with Gaussian Task Embedding and Alignmentcited by 1
  8. 2025Pathological Prior-Guided Multiple Instance Learning For Mitigating Catastrophic Forgetting in Breast Cancer Whole Slide Image Classificationcited by 0
  9. 2025Asynchronous Federated Continual Segmentation with Evolving Clients and Label Spacescited by 0
  10. 2025Harmonizing Generalization and Personalization in Ring-topology Decentralized Federated Learningcited by 0
  11. 2025A Scenario-Oriented Survey of Federated Recommender Systems: Techniques, Challenges, and Future Directionscited by 0
  12. 2025Gains: Fine-grained Federated Domain Adaptation in Open Setcited by 0
  13. 2026Federated Balanced Learningcited by 0
  14. 2026Server-Proximal Aggregation for Federated Domain-Incremental Learning under Partial Participation: Task-Uniform Convergence and Backward Transfercited by 0
  15. 2026Catastrophic Forgetting Resilient One-Shot Incremental Federated Learningcited by 0
  16. 2026FeDMRA: Federated Incremental Learning with Dynamic Memory Replay Allocationcited by 0
  17. 2026From Selection to Scheduling: Federated Geometry-Aware Correction Makes Exemplar Replay Work Better under Continual Dynamic Heterogeneitycited by 0
  18. 2026From Data Heterogeneity to Convergence: A Data-Centric Review of Federated Learningcited by 0
  19. 2026Fisher-Routed Mixture of Experts for Federated Class-Incremental Learningcited by 0
  20. 2026HERO: A Heterogeneity-Aware Benchmark Library for Federated Continual Learningcited by 0
  21. 2026ReSCENE: Server-Side Replay for Structural Mitigation of Catastrophic Forgetting in Federated Continual Learningcited by 0