mlsanityData to 14 Oct 2026

Resource-Constrained Federated Continual Learning: What Does Matter?

arXiv 2501.08737 ↗First posted 15 Jan 2025cs.LGcited by 5

Not on the map. Why?

Abstract (arXiv)

Federated Continual Learning (FCL) aims to enable sequentially privacy-preserving model training on streams of incoming data that vary in edge devices by preserving previous knowledge while adapting to new data. Current FCL literature focuses on restricted data privacy and access to previously seen data while imposing no constraints on the training overhead. This is unreasonable for FCL applications in real-world scenarios, where edge devices are primarily constrained by resources such as storage, computational budget, and label rate. We revisit this problem with a large-scale benchmark and analyze the performance of state-of-the-art FCL approaches under different resource-constrained settings. Various typical FCL techniques and six datasets in two incremental learning scenarios (Class-IL and Domain-IL) are involved in our experiments. Through extensive experiments amounting to a total of over 1,000+ GPU hours, we find that, under limited resource-constrained settings, existing FCL approaches, with no exception, fail to achieve the expected performance. Our conclusions are consistent in the sensitivity analysis. This suggests that most existing FCL methods are particularly too resource-dependent for real-world deployment. Moreover, we study the performance of typical FCL techniques with resource constraints and shed light on future research directions in FCL.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 43 of its 72 references that are in the map. First posted on arXiv 15 Jan 2025.

Builds on

43 of 72 references.

References · 43

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  3. 2015Domain-Adversarial Training of Neural Networkscited by 3,044
  4. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  5. 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,333
  6. 2015Variational Inference with Normalizing Flowscited by 2,170
  7. 2018Experience Replay for Continual Learningcited by 1,095
  8. 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 682
  9. 2019Three scenarios for continual learningcited by 614
  10. 2016Conditional Image Synthesis With Auxiliary Classifier GANscited by 565
  11. 2021Data-Free Knowledge Distillation for Heterogeneous Federated Learningcited by 383
  12. 2017Measuring Catastrophic Forgetting in Neural Networkscited by 360
31 more
  1. 2019Continual learning with hypernetworkscited by 273
  2. 2021Federated Learning for Smart Healthcare: A Surveycited by 184
  3. 2020Revisiting Fundamentals of Experience Replaycited by 175
  4. 2020Federated Continual Learning with Weighted Inter-client Transfercited by 161
  5. 2022Federated Class-Incremental Learningcited by 127
  6. 2021Parameterized Knowledge Transfer for Personalized Federated Learningcited by 109
  7. 2020Continual Learning with Node-Importance based Adaptive Group Sparse Regularizationcited by 105
  8. 2022The CLEAR Benchmark: Continual LEArning on Real-World Imagerycited by 93
  9. 2023TARGET: Federated Class-Continual Learning via Exemplar-Free Distillationcited by 65
  10. 2023Computationally Budgeted Continual Learning: What Does Matter?cited by 61
  11. 2023Better Generative Replay for Continual Federated Learningcited by 55
  12. 2023No One Left Behind: Real-World Federated Class-Incremental Learningcited by 53
  13. 2023Federated Continual Learning via Knowledge Fusion: A Surveycited by 53
  14. 2023A Data-Free Approach to Mitigate Catastrophic Forgetting in Federated Class Incremental Learning for Vision Taskscited by 41
  15. 2023Federated Incremental Semantic Segmentationcited by 36
  16. 2023Cross-Silo Prototypical Calibration for Federated Learning with Non-IID Datacited by 35
  17. 2024Towards Efficient Replay in Federated Incremental Learningcited by 33
  18. 2023FedET: A Communication-Efficient Federated Class-Incremental Learning Framework Based on Enhanced Transformercited by 23
  19. 2024Unleashing the Power of Continual Learning on Non-Centralized Devices: A Surveycited by 23
  20. 2023Federated Orthogonal Training: Mitigating Global Catastrophic Forgetting in Continual Federated Learningcited by 21
  21. 2024Personalized Federated Domain-Incremental Learning based on Adaptive Knowledge Matchingcited by 21
  22. 2024Personalized Federated Continual Learning via Multi-granularity Promptcited by 20
  23. 2023Peer-to-Peer Federated Continual Learning for Naturalistic Driving Action Recognitioncited by 19
  24. 2024Compositional Few-Shot Class-Incremental Learningcited by 18
  25. 2020Impact of ImageNet Model Selection on Domain Adaptationcited by 15
  26. 2024Poisoning Federated Recommender Systems with Fake Userscited by 15
  27. 2024Rehearsal-Free Continual Federated Learning with Synergistic Synaptic Intelligencecited by 15
  28. 2024Federated Continual Learning for Edge-AI: A Comprehensive Surveycited by 11
  29. 2024Delve into Base-Novel Confusion: Redundancy Exploration for Few-Shot Class-Incremental Learningcited by 6
  30. 2024Continual Learning on a Diet: Learning from Sparsely Labeled Streams Under Constrained Computationcited by 4
  31. 2024Better Knowledge Enhancement for Privacy-Preserving Cross-Project Defect Predictioncited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 5.

5 papers.

  1. 2026Accurate and Resource-Efficient Federated Continual Learningcited by 0
  2. 2026Fisher-Routed Mixture of Experts for Federated Class-Incremental Learningcited by 0
  3. 2026HERO: A Heterogeneity-Aware Benchmark Library for Federated Continual Learningcited by 0
  4. 2026Federated Continual Learning as a Distributed Drift-Plus-Penalty Control Problemcited by 0
  5. 2026Federated Subspace Guided Vision-Language-Action Policy Distillation for Non-IID Multi-Robot Manipulationcited by 0