# Streamlined Federated Unlearning: Unite as One to Be Highly Efficient

arXiv: https://arxiv.org/abs/2412.00126 · HTML: https://mlsanity.com/paper/2412.00126/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Federated unlearning and poisoning, one of 12 directions in LLM safety and security.
- Builds on 28 of its 60 references that are in the map.
- First posted on arXiv 28 Nov 2024.

## Placed in

- LLM safety and security / [Federated unlearning and poisoning](https://mlsanity.com/safety/federated-unlearning-and-poisoning/)

## Builds on (most cited first, 28 of 28)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Character-level Convolutional Networks for Text Classification](https://mlsanity.com/paper/1509.01626/) (2015)
- [Machine Unlearning](https://mlsanity.com/paper/1912.03817/) (2019)
- [Bayesian Nonparametric Federated Learning of Neural Networks](https://mlsanity.com/paper/1905.12022/) (2019)
- [Can Bad Teaching Induce Forgetting? Unlearning in Deep Networks using an Incompetent Teacher](https://mlsanity.com/paper/2205.08096/) (2022)
- [Blind Backdoors in Deep Learning Models](https://mlsanity.com/paper/2005.03823/) (2020)
- [Zero-Shot Machine Unlearning](https://mlsanity.com/paper/2201.05629/) (2022)
- [The Right to be Forgotten in Federated Learning: An Efficient Realization with Rapid Retraining](https://mlsanity.com/paper/2203.07320/) (2022)
- [Federated Unlearning via Class-Discriminative Pruning](https://mlsanity.com/paper/2110.11794/) (2021)
- [Federated Unlearning: How to Efficiently Erase a Client in FL?](https://mlsanity.com/paper/2207.05521/) (2022)
- [A Survey on Federated Unlearning: Challenges, Methods, and Future Directions](https://mlsanity.com/paper/2310.20448/) (2023)
- [FedRecover: Recovering from Poisoning Attacks in Federated Learning using Historical Information](https://mlsanity.com/paper/2210.10936/) (2022)
- [VeriFi: Towards Verifiable Federated Unlearning](https://mlsanity.com/paper/2205.12709/) (2022)
- [Federated Unlearning for On-Device Recommendation](https://mlsanity.com/paper/2210.10958/) (2022)
- [Heterogeneous Federated Knowledge Graph Embedding Learning and Unlearning](https://mlsanity.com/paper/2302.02069/) (2023)
- [Communication Efficient and Provable Federated Unlearning](https://mlsanity.com/paper/2401.11018/) (2024)
- [Subspace based Federated Unlearning](https://mlsanity.com/paper/2302.12448/) (2023)
- [Forget-SVGD: Particle-Based Bayesian Federated Unlearning](https://mlsanity.com/paper/2111.12056/) (2021)
- [QuickDrop: Efficient Federated Unlearning by Integrated Dataset Distillation](https://mlsanity.com/paper/2311.15603/) (2023)
- [FedHarmony: Unlearning Scanner Bias with Distributed Data](https://mlsanity.com/paper/2205.15970/) (2022)
- [Towards Efficient and Certified Recovery from Poisoning Attacks in Federated Learning](https://mlsanity.com/paper/2401.08216/) (2024)
- [Compressed Particle-Based Federated Bayesian Learning and Unlearning](https://mlsanity.com/paper/2209.07267/) (2022)
- [Class-wise Federated Unlearning: Harnessing Active Forgetting with Teacher-Student Memory Generation](https://mlsanity.com/paper/2307.03363/) (2023)
- [Privacy-Preserving Autoencoder for Collaborative Object Detection](https://mlsanity.com/paper/2402.18864/) (2024)
- [Learning Privacy-Preserving Student Networks via Discriminative-Generative Distillation](https://mlsanity.com/paper/2409.02404/) (2024)
- [Multitask Identity-Aware Image Steganography via Minimax Optimization](https://mlsanity.com/paper/2107.05819/) (2021)
- [SPFL: A Self-purified Federated Learning Method Against Poisoning Attacks](https://mlsanity.com/paper/2309.10607/) (2023)

## Abstract (arXiv)

Recently, the enactment of ``right to be forgotten" laws and regulations has imposed new privacy requirements on federated learning (FL). Researchers aim to remove the influence of certain data from the trained model without training from scratch through federated unlearning (FU). While current FU research has shown progress in enhancing unlearning efficiency, it often results in degraded model performance upon achieving the goal of data unlearning, necessitating additional steps to recover the performance of the unlearned model. Moreover, these approaches also suffer from many shortcomings such as high consumption of computational and storage resources. To this end, we propose a streamlined federated unlearning approach (SFU) aimed at effectively removing the influence of the target data while preserving the model performance on the retained data without degradation. We design a practical multi-teacher system that achieves both target data influence removal and model performance preservation by guiding the unlearned model through several distinct teacher models. SFU is both computationally and storage-efficient, highly flexible, and generalizable. We conduct extensive experiments on both image and text benchmark datasets. The results demonstrate that SFU significantly improves time and communication efficiency compared to the benchmark retraining method and significantly outperforms existing SOTA methods. Additionally, we verify the effectiveness of SFU using the backdoor attack.