mlsanityData to 7 Oct 2026

Continual Local Training for Better Initialization of Federated Models

arXiv 2005.12657 ↗First posted 26 May 2020cs.LG stat.MLcited by 34

Not on the map. Why?

Abstract (arXiv)

Federated learning (FL) refers to the learning paradigm that trains machine learning models directly in the decentralized systems consisting of smart edge devices without transmitting the raw data, which avoids the heavy communication costs and privacy concerns. Given the typical heterogeneous data distributions in such situations, the popular FL algorithm Federated Averaging (FedAvg) suffers from weight divergence and thus cannot achieve a competitive performance for the global model (denoted as the initial performance in FL) compared to centralized methods. In this paper, we propose the local continual training strategy to address this problem. Importance weights are evaluated on a small proxy dataset on the central server and then used to constrain the local training. With this additional term, we alleviate the weight divergence and continually integrate the knowledge on different local clients into the global model, which ensures a better generalization ability. Experiments on various FL settings demonstrate that our method significantly improves the initial performance of federated models with few extra communication costs.

Key facts. Cited by 34 papers in the map (data through 7 Oct 2026). First posted on arXiv 26 May 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 4; 2022: 7; 2023: 10; 2024: 10; 2025: 1; 2026: 2.

34 papers.

Sort:

  1. 2021Towards Personalized Federated Learningcited by 462
  2. 2023Heterogeneous Federated Learning: State-of-the-art and Research Challengescited by 160
  3. 2023Federated Continual Learning via Knowledge Fusion: A Surveycited by 53
  4. 2021A distillation-based approach integrating continual learning and federated learning for pervasive servicescited by 42
  5. 2023Asynchronous Federated Continual Learningcited by 29
  6. 2023Privacy and Fairness in Federated Learning: on the Perspective of Trade-offcited by 26
  7. 2022Federated Learning from Only Unlabeled Data with Class-Conditional-Sharing Clientscited by 20
  8. 2022SphereFed: Hyperspherical Federated Learningcited by 19
  9. 2023FedBone: Towards Large-Scale Federated Multi-Task Learningcited by 14
  10. 2022Spatial-Temporal Federated Learning for Lifelong Person Re-identification on Distributed Edgescited by 12
  11. 2024Advances in Robust Federated Learning: A Survey with Heterogeneity Considerationscited by 12
  12. 2022FedKNOW: Federated Continual Learning with Signature Task Knowledge Integration at Edgecited by 10
22 more
  1. 2025BrainGuard: Privacy-Preserving Multisubject Image Reconstructions from Brain Activitiescited by 10
  2. 2022Continual Horizontal Federated Learning for Heterogeneous Datacited by 8
  3. 2024A Comprehensive Survey of Federated Transfer Learning: Challenges, Methods and Applicationscited by 8
  4. 2024Free Lunch for Federated Remote Sensing Target Fine-Grained Classification: A Parameter-Efficient Frameworkcited by 6
  5. 2023Jointly Exploring Client Drift and Catastrophic Forgetting in Dynamic Learningcited by 5
  6. 2023FedJudge: Federated Legal Large Language Modelcited by 5
  7. 2021Bristle: Decentralized Federated Learning in Byzantine, Non-i.i.d. Environmentscited by 4
  8. 2023A Survey of Incremental Transfer Learning: Combining Peer-to-Peer Federated Learning and Domain Incremental Learning for Multicenter Collaborationcited by 4
  9. 2024Federated Distillation for Medical Image Classification: Towards Trustworthy Computer-Aided Diagnosiscited by 4
  10. 2023On Knowledge Editing in Federated Learning: Perspectives, Challenges, and Future Directionscited by 3
  11. 2023Coordinated Replay Sample Selection for Continual Federated Learningcited by 3
  12. 2022Global Update Guided Federated Learningcited by 2
  13. 2024FedMeS: Personalized Federated Continual Learning Leveraging Local Memorycited by 1
  14. 2024Buffer-based Gradient Projection for Continual Federated Learningcited by 1
  15. 2024FedRewind: Rewinding Continual Model Exchange for Decentralized Federated Learningcited by 1
  16. 2021Towards More Efficient Federated Learning with Better Optimization Objectscited by 0
  17. 2022Exploration and Exploitation in Federated Learning to Exclude Clients with Poisoned Datacited by 0
  18. 2024Federated Learning with New Knowledge: Fundamentals, Advances, and Futurescited by 0
  19. 2024Bridging Data Barriers among Participants: Assessing the Potential of Geoenergy through Federated Learningcited by 0
  20. 2024Fast convergence of a Federated Expectation-Maximization Algorithmcited by 0
  21. 2026Multi-Level Analyzation of Imbalance to Resolve Non-IID-Ness in Federated Learningcited by 0
  22. 2026An Inclusive and Lightweight Approach to Federated Continual Learning for Cultural Heritagecited by 0