# Federated Unlearning with Knowledge Distillation arXiv: https://arxiv.org/abs/2201.09441 ยท HTML: https://mlsanity.com/paper/2201.09441/ ## Key facts - Cited by 73 papers in the map (data through 14 Oct 2026). - Placed in Federated unlearning and poisoning, one of 12 directions in LLM safety and security. - Builds on 13 of its 29 references that are in the map. - First posted on arXiv 24 Jan 2022. ## Placed in - LLM safety and security / [Federated unlearning and poisoning](https://mlsanity.com/safety/federated-unlearning-and-poisoning/) - Knowledge distillation / [Federated, robust and private distillation](https://mlsanity.com/distill/federated-robust-and-private-distillation/) ## Builds on (most cited first, 13 of 13) - [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014) - [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015) - [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016) - [BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain](https://mlsanity.com/paper/1708.06733/) (2017) - [Machine Unlearning](https://mlsanity.com/paper/1912.03817/) (2019) - [How To Backdoor Federated Learning](https://mlsanity.com/paper/1807.00459/) (2018) - [Distillation as a Defense to Adversarial Perturbations against Deep Neural Networks](https://mlsanity.com/paper/1511.04508/) (2015) - [ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Models](https://mlsanity.com/paper/1806.01246/) (2018) - [Making AI Forget You: Data Deletion in Machine Learning](https://mlsanity.com/paper/1907.05012/) (2019) - [Analyzing Federated Learning through an Adversarial Lens](https://mlsanity.com/paper/1811.12470/) (2018) - [Approximate Data Deletion from Machine Learning Models](https://mlsanity.com/paper/2002.10077/) (2020) - [The Secret Revealer: Generative Model-Inversion Attacks Against Deep Neural Networks](https://mlsanity.com/paper/1911.07135/) (2019) - [Detecting Cancer Metastases on Gigapixel Pathology Images](https://mlsanity.com/paper/1703.02442/) (2017) ## Built on by (most cited first, 30 of 73) - [A Survey of Machine Unlearning](https://mlsanity.com/paper/2209.02299/) (2022) - [Towards Unbounded Machine Unlearning](https://mlsanity.com/paper/2302.09880/) (2023) - [Can Bad Teaching Induce Forgetting? Unlearning in Deep Networks using an Incompetent Teacher](https://mlsanity.com/paper/2205.08096/) (2022) - [Federated Unlearning: How to Efficiently Erase a Client in FL?](https://mlsanity.com/paper/2207.05521/) (2022) - [A Survey on Federated Unlearning: Challenges, Methods, and Future Directions](https://mlsanity.com/paper/2310.20448/) (2023) - [Scissorhands: Scrub Data Influence via Connection Sensitivity in Networks](https://mlsanity.com/paper/2401.06187/) (2024) - [Exploring the Landscape of Machine Unlearning: A Comprehensive Survey and Taxonomy](https://mlsanity.com/paper/2305.06360/) (2023) - [Threats, Attacks, and Defenses in Machine Unlearning: A Survey](https://mlsanity.com/paper/2403.13682/) (2024) - [Machine Unlearning: A Comprehensive Survey](https://mlsanity.com/paper/2405.07406/) (2024) - [Backdoor Attacks and Defenses in Federated Learning: Survey, Challenges and Future Research Directions](https://mlsanity.com/paper/2303.02213/) (2023) - [Deep Regression Unlearning](https://mlsanity.com/paper/2210.08196/) (2022) - [A Review on Machine Unlearning](https://mlsanity.com/paper/2411.11315/) (2024) - [Knowledge Distillation for Federated Learning: a Practical Guide](https://mlsanity.com/paper/2211.04742/) (2022) - [Federated Unlearning: A Survey on Methods, Design Guidelines, and Evaluation Metrics](https://mlsanity.com/paper/2401.05146/) (2024) - [Machine Unlearning of Federated Clusters](https://mlsanity.com/paper/2210.16424/) (2022) - [Machine Unlearning: A Survey](https://mlsanity.com/paper/2306.03558/) (2023) - [Learning to Unlearn while Retaining: Combating Gradient Conflicts in Machine Unlearning](https://mlsanity.com/paper/2503.06339/) (2025) - [Data and Model Poisoning Backdoor Attacks on Wireless Federated Learning, and the Defense Mechanisms: A Comprehensive Survey](https://mlsanity.com/paper/2312.08667/) (2023) - [Subspace based Federated Unlearning](https://mlsanity.com/paper/2302.12448/) (2023) - [MUNBa: Machine Unlearning via Nash Bargaining](https://mlsanity.com/paper/2411.15537/) (2024) - [A Survey on Machine Unlearning: Techniques and New Emerged Privacy Risks](https://mlsanity.com/paper/2406.06186/) (2024) - [UnStar: Unlearning with Self-Taught Anti-Sample Reasoning for LLMs](https://mlsanity.com/paper/2410.17050/) (2024) - [NoT: Federated Unlearning via Weight Negation](https://mlsanity.com/paper/2503.05657/) (2025) - [Unlearning during Learning: An Efficient Federated Machine Unlearning Method](https://mlsanity.com/paper/2405.15474/) (2024) - [Get Rid Of Your Trail: Remotely Erasing Backdoors in Federated Learning](https://mlsanity.com/paper/2304.10638/) (2023) - [Distill to Delete: Unlearning in Graph Networks with Knowledge Distillation](https://mlsanity.com/paper/2309.16173/) (2023) - [Ferrari: Federated Feature Unlearning via Optimizing Feature Sensitivity](https://mlsanity.com/paper/2405.17462/) (2024) - [Enable the Right to be Forgotten with Federated Client Unlearning in Medical Imaging](https://mlsanity.com/paper/2407.02356/) (2024) - [Fast-FedUL: A Training-Free Federated Unlearning with Provable Skew Resilience](https://mlsanity.com/paper/2405.18040/) (2024) - [Federated Unlearning: a Perspective of Stability and Fairness](https://mlsanity.com/paper/2402.01276/) (2024) ## Abstract (arXiv) Federated Learning (FL) is designed to protect the data privacy of each client during the training process by transmitting only models instead of the original data. However, the trained model may memorize certain information about the training data. With the recent legislation on right to be forgotten, it is crucially essential for the FL model to possess the ability to forget what it has learned from each client. We propose a novel federated unlearning method to eliminate a client's contribution by subtracting the accumulated historical updates from the model and leveraging the knowledge distillation method to restore the model's performance without using any data from the clients. This method does not have any restrictions on the type of neural networks and does not rely on clients' participation, so it is practical and efficient in the FL system. We further introduce backdoor attacks in the training process to help evaluate the unlearning effect. Experiments on three canonical datasets demonstrate the effectiveness and efficiency of our method.