mlsanityData to 7 Oct 2026

Towards Federated Learning on Time-Evolving Heterogeneous Data

arXiv 2112.13246 ↗First posted 25 Dec 2021cs.LGcited by 23

Not on the map. Why?

Abstract (arXiv)

Federated Learning (FL) is a learning paradigm that protects privacy by keeping client data on edge devices. However, optimizing FL in practice can be difficult due to the diversity and heterogeneity of the learning system. Despite recent research efforts to improve the optimization of heterogeneous data, the impact of time-evolving heterogeneous data in real-world scenarios, such as changing client data or intermittent clients joining or leaving during training, has not been studied well. In this work, we propose Continual Federated Learning (CFL), a flexible framework for capturing the time-evolving heterogeneity of FL. CFL can handle complex and realistic scenarios, which are difficult to evaluate in previous FL formulations, by extracting information from past local data sets and approximating local objective functions. We theoretically demonstrate that CFL methods have a faster convergence rate than FedAvg in time-evolving scenarios, with the benefit depending on approximation quality. Through experiments, we show that our numerical findings match the convergence analysis and that CFL methods significantly outperform other state-of-the-art FL baselines.

Key facts. Cited by 23 papers in the map (data through 7 Oct 2026). Builds on 44 of its 61 references that are in the map. First posted on arXiv 25 Dec 2021.

Builds on

44 of 61 references.

References · 44

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  3. 2019Advances and Open Problems in Federated Learningcited by 3,021
  4. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  5. 2016Learning without Forgettingcited by 2,465
  6. 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,313
  7. 2016Progressive Neural Networkscited by 1,332
  8. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,220
  9. 2020Adaptive Federated Optimizationcited by 1,036
  10. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 897
  11. 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 836
  12. 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 678
32 more
  1. 2020Ditto: Fair and Robust Federated Learning Through Personalizationcited by 665
  2. 2020Personalized Federated Learning with Moreau Envelopescited by 612
  3. 2018End-to-End Incremental Learningcited by 596
  4. 2018Local SGD Converges Fast and Communicates Littlecited by 595
  5. 2021Exploiting Shared Representations for Personalized Federated Learningcited by 537
  6. 2019Agnostic Federated Learningcited by 451
  7. 2019Fair Resource Allocation in Federated Learningcited by 389
  8. 2019Bayesian Nonparametric Federated Learning of Neural Networkscited by 385
  9. 2020A Unified Theory of Decentralized SGD with Changing Topology and Local Updatescited by 361
  10. 2020HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clientscited by 334
  11. 2019PyHessian: Neural Networks Through the Lens of the Hessiancited by 291
  12. 2019Tighter Theory for Local SGD on Identical and Heterogeneous Datacited by 266
  13. 2021No Fear of Heterogeneity: Classifier Calibration for Federated Learning with Non-IID Datacited by 241
  14. 2020Federated Learning of a Mixture of Global and Local Modelscited by 208
  15. 2019SGD: General Analysis and Improved Ratescited by 206
  16. 2018Don't Use Large Mini-Batches, Use Local SGDcited by 201
  17. 2020Federated Learning with Compression: Unified Analysis and Sharp Guaranteescited by 159
  18. 2020Federated Continual Learning with Weighted Inter-client Transfercited by 158
  19. 2020Mime: Mimicking Centralized Stochastic Algorithms in Federated Learningcited by 152
  20. 2020Is Local SGD Better than Minibatch SGD?cited by 151
  21. 2020Minibatch vs Local SGD for Heterogeneous Distributed Learningcited by 147
  22. 2019Overcoming Forgetting in Federated Learning on Non-IID Datacited by 120
  23. 2019Which Algorithmic Choices Matter at Which Batch Sizes? Insights From a Noisy Quadratic Modelcited by 116
  24. 2018Continuous Learning in Single-Incremental-Task Scenarioscited by 109
  25. 2020Robust Federated Learning: The Case of Affine Distribution Shiftscited by 102
  26. 2020Faster Non-Convex Federated Learning via Global and Local Momentumcited by 48
  27. 2021A distillation-based approach integrating continual learning and federated learning for pervasive servicescited by 42
  28. 2021Concept drift detection and adaptation for federated and continual learningcited by 32
  29. 2018Partitioned Variational Inference: A unified framework encompassing federated and continual learningcited by 29
  30. 2021On Bridging Generic and Personalized Federated Learning for Image Classificationcited by 15
  31. 2020Optimization and Generalization of Regularization-Based Continual Learning: a Loss Approximation Viewpointcited by 10
  32. 2019Communication trade-offs for synchronized distributed SGD with large step sizecited by 4

Built on by

Cited by year

Citing papers per year: 2022: 5; 2023: 8; 2024: 7; 2025: 3.

23 papers.

Sort:

By area:

  1. 2023Federated Continual Learning via Knowledge Fusion: A Surveycited by 53
  2. 2022Test-Time Robust Personalization for Federated Learningcited by 39
  3. 2022FedBR: Improving Federated Learning on Heterogeneous Data via Local Learning Bias Reductioncited by 35
  4. 2022Federated Learning under Distributed Concept Driftcited by 35
  5. 2022DELTA: Diverse Client Sampling for Fasting Federated Learningcited by 24
  6. 2024Unleashing the Power of Continual Learning on Non-Centralized Devices: A Surveycited by 23
  7. 2023Federated Learning for Data Streamscited by 15
  8. 2023FedRC: Tackling Diverse Distribution Shifts Challenge in Federated Learning by Robust Clusteringcited by 15
  9. 2023Performative Federated Learning: A Solution to Model-Dependent and Heterogeneous Distribution Shiftscited by 14
  10. 2024Non-IID data in Federated Learning: A Survey with Taxonomy, Metrics, Methods, Frameworks and Future Directionscited by 12
  11. 2025FedRSClip: Federated Learning for Remote Sensing Scene Classification Using Vision-Language Modelscited by 11
  12. 2024Federated Continual Learning for Edge-AI: A Comprehensive Surveycited by 10
11 more
  1. 2022Online Federated Learning via Non-Stationary Detection and Adaptation amidst Concept Driftcited by 9
  2. 2024Classifier Clustering and Feature Alignment for Federated Learning under Distributed Concept Driftcited by 8
  3. 2023Jointly Exploring Client Drift and Catastrophic Forgetting in Dynamic Learningcited by 5
  4. 2023Enhancing Clustered Federated Learning: Integration of Strategies and Improved Methodologiescited by 3
  5. 2023Coordinated Replay Sample Selection for Continual Federated Learningcited by 3
  6. 2025Towards Heterogeneity-Aware and Energy-Efficient Topology Optimization for Decentralized Federated Learning in Edge Environmentcited by 3
  7. 2023Concept Matching: Clustering-based Federated Continual Learningcited by 1
  8. 2024CurvFed: Curvature-Aligned Federated Learning for Fairness without Demographicscited by 1
  9. 2024ProFL: Performative Robust Optimal Federated Learningcited by 1
  10. 2024Client2Vec: Improving Federated Learning by Distribution Shifts Aware Client Indexingcited by 0
  11. 2025FedEve: On Bridging the Client Drift and Period Drift for Cross-device Federated Learningcited by 0