FedMeS: Personalized Federated Continual Learning Leveraging Local Memory
Not on the map. Why?
Abstract (arXiv)
We focus on the problem of Personalized Federated Continual Learning (PFCL): a group of distributed clients, each with a sequence of local tasks on arbitrary data distributions, collaborate through a central server to train a personalized model at each client, with the model expected to achieve good performance on all local tasks. We propose a novel PFCL framework called Federated Memory Strengthening FedMeS to address the challenges of client drift and catastrophic forgetting. In FedMeS, each client stores samples from previous tasks using a small amount of local memory, and leverages this information to both 1) calibrate gradient updates in training process; and 2) perform KNN-based Gaussian inference to facilitate personalization. FedMeS is designed to be task-oblivious, such that the same inference process is applied to samples from all tasks to achieve good performance. FedMeS is analyzed theoretically and evaluated experimentally. It is shown to outperform all baselines in average accuracy and forgetting rate, over various combinations of datasets, task distributions, and client numbers.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 32 of its 51 references that are in the map. First posted on arXiv 19 Apr 2024.
Builds on
32 of 51 references.
References · 32
- 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
- 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
- 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,313
- 2016Matching Networks for One Shot Learningcited by 2,246
- 2017Gradient Episodic Memory for Continual Learningcited by 1,871
- 2017Continual Learning with Deep Generative Replaycited by 1,152
- 2016Optimization Methods for Large-Scale Machine Learningcited by 1,140
- 2018Efficient Lifelong Learning with A-GEMcited by 1,044
- 2019Generalization through Memorization: Nearest Neighbor Language Modelscited by 788
- 2020Ditto: Fair and Robust Federated Learning Through Personalizationcited by 665
- 2018Learning to Learn without Forgetting by Maximizing Transfer and Minimizing Interferencecited by 511
- 2020Adaptive Personalized Federated Learningcited by 353
20 more
- 2020Three Approaches for Personalization with Applications to Federated Learningcited by 321
- 2017CORe50: a New Dataset and Benchmark for Continuous Object Recognitioncited by 287
- 2020Personalized Cross-Silo Federated Learning on Non-IID Datacited by 282
- 2020Semantic Drift Compensation for Class-Incremental Learningcited by 241
- 2018Selective Experience Replay for Lifelong Learningcited by 239
- 2021Federated Multi-Task Learning under a Mixture of Distributionscited by 189
- 2019Adaptive Gradient-Based Meta-Learning Methodscited by 183
- 2022Fine-tuning Global Model via Data-Free Knowledge Distillation for Non-IID Federated Learningcited by 172
- 2020Federated Continual Learning with Weighted Inter-client Transfercited by 158
- 2020Salvaging Federated Learning by Local Adaptationcited by 142
- 2019Overcoming Forgetting in Federated Learning on Non-IID Datacited by 120
- 2022Memory Replay with Data Compression for Continual Learningcited by 80
- 2021Personalized Federated Learning with Gaussian Processescited by 76
- 2020Federated Mutual Learningcited by 70
- 2021Personalized Federated Learning through Local Memorizationcited by 57
- 2021A distillation-based approach integrating continual learning and federated learning for pervasive servicescited by 42
- 2020Continual Local Training for Better Initialization of Federated Modelscited by 34
- 2021Continual Learning via Bit-Level Information Preservingcited by 32
- 2022FedKNOW: Federated Continual Learning with Signature Task Knowledge Integration at Edgecited by 10
- 2022Addressing Client Drift in Federated Continual Learning with Adaptive Optimizationcited by 5
Built on by
Citing papers per year: 2024: 1.
1 paper.