mlsanityData to 7 Oct 2026

Federated Unlearning with Knowledge Distillation

arXiv 2201.09441 ↗First posted 24 Jan 2022cs.LG cs.CRcited by 70

On the map: LLM safety and security → Federated unlearning and poisoning

Abstract (arXiv)

Federated Learning (FL) is designed to protect the data privacy of each client during the training process by transmitting only models instead of the original data. However, the trained model may memorize certain information about the training data. With the recent legislation on right to be forgotten, it is crucially essential for the FL model to possess the ability to forget what it has learned from each client. We propose a novel federated unlearning method to eliminate a client's contribution by subtracting the accumulated historical updates from the model and leveraging the knowledge distillation method to restore the model's performance without using any data from the clients. This method does not have any restrictions on the type of neural networks and does not rely on clients' participation, so it is practical and efficient in the FL system. We further introduce backdoor attacks in the training process to help evaluate the unlearning effect. Experiments on three canonical datasets demonstrate the effectiveness and efficiency of our method.

Key facts. Cited by 70 papers in the map (data through 7 Oct 2026). Placed in Federated unlearning and poisoning, one of 12 directions in LLM safety and security. Builds on 13 of its 29 references that are in the map. First posted on arXiv 24 Jan 2022.

Builds on

13 of 29 references.

References · 13

  1. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  2. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  3. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  4. 2017BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chaincited by 1,052
  5. 2019Machine Unlearningcited by 1,020
  6. 2018How To Backdoor Federated Learningcited by 745
  7. 2015Distillation as a Defense to Adversarial Perturbations against Deep Neural Networkscited by 675
  8. 2018ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Modelscited by 475
  9. 2019Making AI Forget You: Data Deletion in Machine Learningcited by 447
  10. 2018Analyzing Federated Learning through an Adversarial Lenscited by 346
  11. 2020Approximate Data Deletion from Machine Learning Modelscited by 250
  12. 2019The Secret Revealer: Generative Model-Inversion Attacks Against Deep Neural Networkscited by 241
1 more
  1. 2017Detecting Cancer Metastases on Gigapixel Pathology Imagescited by 51

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 6; 2023: 12; 2024: 30; 2025: 11; 2026: 11.

70 papers.

Sort:

By area:

  1. 2022A Survey of Machine Unlearningcited by 298
  2. 2023Towards Unbounded Machine Unlearningcited by 261
  3. 2022Can Bad Teaching Induce Forgetting? Unlearning in Deep Networks using an Incompetent Teachercited by 189
  4. 2022Federated Unlearning: How to Efficiently Erase a Client in FL?cited by 90
  5. 2024Scissorhands: Scrub Data Influence via Connection Sensitivity in Networkscited by 50
  6. 2023Exploring the Landscape of Machine Unlearning: A Comprehensive Survey and Taxonomycited by 44
  7. 2024Threats, Attacks, and Defenses in Machine Unlearning: A Surveycited by 34
  8. 2023Backdoor Attacks and Defenses in Federated Learning: Survey, Challenges and Future Research Directionscited by 33
  9. 2024Machine Unlearning: A Comprehensive Surveycited by 33
  10. 2022Deep Regression Unlearningcited by 32
  11. 2022Knowledge Distillation for Federated Learning: a Practical Guidecited by 31
  12. 2024A Review on Machine Unlearningcited by 30
38 more
  1. 2024Federated Unlearning: A Survey on Methods, Design Guidelines, and Evaluation Metricscited by 25
  2. 2022Machine Unlearning of Federated Clusterscited by 21
  3. 2025Learning to Unlearn while Retaining: Combating Gradient Conflicts in Machine Unlearningcited by 19
  4. 2023Machine Unlearning: A Surveycited by 18
  5. 2023Data and Model Poisoning Backdoor Attacks on Wireless Federated Learning, and the Defense Mechanisms: A Comprehensive Surveycited by 17
  6. 2024MUNBa: Machine Unlearning via Nash Bargainingcited by 17
  7. 2025NoT: Federated Unlearning via Weight Negationcited by 15
  8. 2024Unlearning during Learning: An Efficient Federated Machine Unlearning Methodcited by 14
  9. 2024UnStar: Unlearning with Self-Taught Anti-Sample Reasoning for LLMscited by 14
  10. 2023Subspace based Federated Unlearningcited by 13
  11. 2023Distill to Delete: Unlearning in Graph Networks with Knowledge Distillationcited by 13
  12. 2023Get Rid Of Your Trail: Remotely Erasing Backdoors in Federated Learningcited by 12
  13. 2024Ferrari: Federated Feature Unlearning via Optimizing Feature Sensitivitycited by 11
  14. 2024Enable the Right to be Forgotten with Federated Client Unlearning in Medical Imagingcited by 9
  15. 2024Fast-FedUL: A Training-Free Federated Unlearning with Provable Skew Resiliencecited by 8
  16. 2024Federated Unlearning: a Perspective of Stability and Fairnesscited by 7
  17. 2024Multi-Modal Recommendation Unlearning for Legal, Licensing, and Modality Constraintscited by 5
  18. 2024Towards Privacy-Guaranteed Label Unlearning in Vertical Federated Learning: Few-Shot Forgetting without Disclosurecited by 5
  19. 2026EASE: Federated Multimodal Unlearning via Entanglement-Aware Anchor Closurecited by 5
  20. 2023Defending Our Privacy With Backdoorscited by 4
  21. 2024How to Forget Clients in Federated Online Learning to Rank?cited by 4
  22. 2024Vertical Federated Unlearning via Backdoor Certificationcited by 4
  23. 2025Deep Contrastive Unlearning for Language Modelscited by 4
  24. 2023On Knowledge Editing in Federated Learning: Perspectives, Challenges, and Future Directionscited by 3
  25. 2023Heterogeneous Decentralized Machine Unlearning with Seed Model Distillationcited by 3
  26. 2024BlockFUL: Enabling Unlearning in Blockchained Federated Learningcited by 3
  27. 2024Goldfish: An Efficient Federated Unlearning Frameworkcited by 3
  28. 2024FedUHB: Accelerating Federated Unlearning via Polyak Heavy Ball Methodcited by 3
  29. 2024Machine unlearning through fine-grained model parameters perturbationcited by 2
  30. 2024FedQUIT: On-Device Federated Unlearning via a Quasi-Competent Virtual Teachercited by 2
  31. 2024ConDa: Fast Federated Unlearning with Contribution Dampeningcited by 2
  32. 2024FedBlock: A Blockchain Approach to Federated Learning against Backdoor Attackscited by 2
  33. 2025Exploring Incremental Unlearning: Techniques, Challenges, and Future Directionscited by 2
  34. 2025Quantum-Inspired Audio Unlearning: Towards Privacy-Preserving Voice Biometricscited by 2
  35. 2023Machine Unlearning Methodology base on Stochastic Teacher Networkcited by 1
  36. 2024Federated Unlearning for Human Activity Recognitioncited by 1
  37. 2024Efficient Federated Unlearning under Plausible Deniabilitycited by 1
  38. 2025Federated Unlearning Made Practical: Seamless Integration via Negated Pseudo-Gradientscited by 1

50 most cited of 70 All 70 (JSON)