mlsanityData to 7 Oct 2026

Server-Proximal Aggregation for Federated Domain-Incremental Learning under Partial Participation: Task-Uniform Convergence and Backward Transfer

arXiv 2601.22274 ↗First posted 29 Jan 2026cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Real-world federated systems seldom operate on static data: input distributions drift while privacy rules forbid raw-data sharing. We study this setting as Federated Domain-Incremental Learning (FDIL), where (i) clients are heterogeneous, (ii) tasks arrive sequentially with shifting domains, yet (iii) the label space remains fixed. Two theoretical pillars remain missing for FDIL under realistic deployment: a guarantee of backward knowledge transfer (BKT) and a convergence rate that holds across the sequence of all tasks with partial participation. We introduce SPECIAL (Server-Proximal Efficient Continual Aggregation for Learning), a simple, memory-free FDIL algorithm that adds a single server-side ``anchor'' to vanilla FedAvg: in each round, the server nudges the uniformly sampled participated clients update toward the previous global model with a lightweight proximal term. This anchor curbs cumulative drift without replay buffers, synthetic data, or task-specific heads, keeping communication and model size unchanged. Our theory shows that SPECIAL (i) preserves earlier tasks: a BKT bound caps any increase in prior-task loss by a drift-controlled term that shrinks with more rounds, local epochs, and participating clients; and (ii) learns efficiently across all tasks: the first communication-efficient non-convex convergence rate for FDIL with partial participation, O((E/NT)^(1/2)), with E local epochs, T communication rounds, and N participated clients per round, matching single-task FedAvg while explicitly separating optimization variance from inter-task drift. Experimental results further demonstrate the effectiveness of SPECIAL.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 42 of its 54 references that are in the map. First posted on arXiv 29 Jan 2026.

Builds on

42 of 54 references.

References · 42

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  3. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  4. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  5. 2016Learning without Forgettingcited by 2,465
  6. 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,313
  7. 2017Gradient Episodic Memory for Continual Learningcited by 1,871
  8. 2018Continual Lifelong Learning with Neural Networks: A Reviewcited by 1,313
  9. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,220
  10. 2017Memory Aware Synapses: Learning what (not) to forgetcited by 1,094
  11. 2020Adaptive Federated Optimizationcited by 1,036
  12. 2017Deeper, Broader and Artier Domain Generalizationcited by 983
30 more
  1. 2023A Comprehensive Survey of Continual Learning: Theory, Method and Applicationcited by 860
  2. 2019Large Scale Incremental Learningcited by 727
  3. 2018Local SGD Converges Fast and Communicates Littlecited by 595
  4. 2018Progress & Compress: A scalable framework for continual learningcited by 516
  5. 2018Learning to Learn without Forgetting by Maximizing Transfer and Minimizing Interferencecited by 511
  6. 2019Learn to Grow: A Continual Structure Learning Framework for Overcoming Catastrophic Forgettingcited by 302
  7. 2021Achieving Linear Speedup with Partial Worker Participation in Non-IID Federated Learningcited by 192
  8. 2019Meta-Learning Representations for Continual Learningcited by 189
  9. 2020Federated Continual Learning with Weighted Inter-client Transfercited by 158
  10. 2014A Proximal Stochastic Gradient Method with Progressive Variance Reductioncited by 78
  11. 2023TARGET: Federated Class-Continual Learning via Exemplar-Free Distillationcited by 64
  12. 2022Beyond Not-Forgetting: Continual Learning with Backward Knowledge Transfercited by 59
  13. 2023Better Generative Replay for Continual Federated Learningcited by 53
  14. 2021A distillation-based approach integrating continual learning and federated learning for pervasive servicescited by 42
  15. 2023A Data-Free Approach to Mitigate Catastrophic Forgetting in Federated Class Incremental Learning for Vision Taskscited by 41
  16. 2024Towards Efficient Replay in Federated Incremental Learningcited by 33
  17. 2023A Unified Approach to Domain Incremental Learning with Memory: Theory and Algorithmcited by 32
  18. 2021Elastic Weight Consolidation (EWC): Nuts and Boltscited by 21
  19. 2024Personalized Federated Domain-Incremental Learning based on Adaptive Knowledge Matchingcited by 21
  20. 2022A Theory for Knowledge Transfer in Continual Learningcited by 16
  21. 2023Federated Learning for Data Streamscited by 15
  22. 2025Accurate Forgetting for Heterogeneous Federated Continual Learningcited by 11
  23. 2022Federated Continual Learning through distillation in pervasive computingcited by 10
  24. 2022Towards Efficient and Stable K-Asynchronous Federated Learning with Unbounded Stale Gradients on Non-IID Datacited by 9
  25. 2024On the Convergence of Continual Learning with Adaptive Methodscited by 7
  26. 2023Dual Cognitive Architecture: Incorporating Biases and Multi-Memory Systems for Lifelong Learningcited by 6
  27. 2024On the Convergence of Continual Federated Learning Using Incrementally Aggregated Gradientscited by 5
  28. 2023Don't Memorize; Mimic The Past: Federated Class Incremental Learning Without Episodic Memorycited by 4
  29. 2024Rehearsal-free Federated Domain-incremental Learningcited by 4
  30. 2023CoDeC: Communication-Efficient Decentralized Continual Learningcited by 2

Built on by

Not cited yet.