On Knowledge Editing in Federated Learning: Perspectives, Challenges, and Future Directions
Not on the map. Why?
Abstract (arXiv)
As Federated Learning (FL) has gained increasing attention, it has become widely acknowledged that straightforwardly applying stochastic gradient descent (SGD) on the overall framework when learning over a sequence of tasks results in the phenomenon known as ``catastrophic forgetting''. Consequently, much FL research has centered on devising federated increasing learning methods to alleviate forgetting while augmenting knowledge. On the other hand, forgetting is not always detrimental. The selective amnesia, also known as federated unlearning, which entails the elimination of specific knowledge, can address privacy concerns and create additional ``space'' for acquiring new knowledge. However, there is a scarcity of extensive surveys that encompass recent advancements and provide a thorough examination of this issue. In this manuscript, we present an extensive survey on the topic of knowledge editing (augmentation/removal) in Federated Learning, with the goal of summarizing the state-of-the-art research and expanding the perspective for various domains. Initially, we introduce an integrated paradigm, referred to as Federated Editable Learning (FEL), by reevaluating the entire lifecycle of FL. Secondly, we provide a comprehensive overview of existing methods, evaluate their position within the proposed paradigm, and emphasize the current challenges they face. Lastly, we explore potential avenues for future research and identify unresolved issues.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 41 of its 76 references that are in the map. First posted on arXiv 2 Jun 2023.
Builds on
41 of 76 references.
References · 41
- 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
- 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
- 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
- 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,313
- 2016Membership Inference Attacks against Machine Learning Modelscited by 2,221
- 2014Deep Learning in Neural Networks: An Overviewcited by 765
- 2018How To Backdoor Federated Learningcited by 745
- 2019Large Scale Incremental Learningcited by 727
- 2020How Much Knowledge Can You Pack Into the Parameters of a Language Model?cited by 713
- 2013An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networkscited by 711
- 2020The Future of Digital Health with Federated Learningcited by 625
- 2020Class-incremental learning: survey and performance evaluation on image classificationcited by 446
29 more
- 2019A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protectioncited by 371
- 2020Attack of the Tails: Yes, You Really Can Backdoor Federated Learningcited by 307
- 2019Can You Really Backdoor Federated Learning?cited by 250
- 2019FedHealth: A Federated Transfer Learning Framework for Wearable Healthcarecited by 182
- 2021Adaptive Machine Unlearningcited by 164
- 2020Federated Continual Learning with Weighted Inter-client Transfercited by 158
- 2018Federated Meta-Learning with Fast Convergence and Efficient Communicationcited by 154
- 2022Federated Class-Incremental Learningcited by 126
- 2019Overcoming Forgetting in Federated Learning on Non-IID Datacited by 120
- 2021FairFed: Enabling Group Fairness in Federated Learningcited by 114
- 2022The Right to be Forgotten in Federated Learning: An Efficient Realization with Rapid Retrainingcited by 109
- 2021Federated Unlearning via Class-Discriminative Pruningcited by 108
- 2022Federated Unlearning: How to Efficiently Erase a Client in FL?cited by 90
- 2020Machine Unlearning: Linear Filtration for Logit-based Classifierscited by 89
- 2021Proof-of-Learning: Definitions and Practicecited by 81
- 2022Federated Unlearning with Knowledge Distillationcited by 70
- 2022FedRecover: Recovering from Poisoning Attacks in Federated Learning using Historical Informationcited by 53
- 2021FLOP: Federated Learning on Medical Datasets using Partial Networkscited by 42
- 2022A Survey on Gradient Inversion: Attacks, Defenses and Future Directionscited by 39
- 2020Continual Local Training for Better Initialization of Federated Modelscited by 34
- 2021Concept drift detection and adaptation for federated and continual learningcited by 32
- 2021Source Inference Attacks in Federated Learningcited by 31
- 2022Disentangled Federated Learning for Tackling Attributes Skew via Invariant Aggregation and Diversity Transferringcited by 28
- 2021Federated Reconnaissance: Efficient, Distributed, Class-Incremental Learningcited by 14
- 2021Forget-SVGD: Particle-Based Bayesian Federated Unlearningcited by 14
- 2021Federated Few-Shot Learning with Adversarial Learningcited by 11
- 2022Federated Continual Learning through distillation in pervasive computingcited by 10
- 2022LegoNet: A Fast and Exact Unlearning Architecturecited by 6
- 2022FewFedWeight: Few-shot Federated Learning Framework across Multiple NLP Taskscited by 3
Built on by
Citing papers per year: 2024: 3.
3 papers.