# Unlearning during Learning: An Efficient Federated Machine Unlearning Method

arXiv: https://arxiv.org/abs/2405.15474 · HTML: https://mlsanity.com/paper/2405.15474/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Placed in Federated unlearning and poisoning, one of 12 directions in LLM safety and security.
- Builds on 18 of its 34 references that are in the map.
- First posted on arXiv 24 May 2024.

## Placed in

- LLM safety and security / [Federated unlearning and poisoning](https://mlsanity.com/safety/federated-unlearning-and-poisoning/)

## Builds on (most cited first, 18 of 18)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Machine Unlearning](https://mlsanity.com/paper/1912.03817/) (2019)
- [Federated Learning on Non-IID Data Silos: An Experimental Study](https://mlsanity.com/paper/2102.02079/) (2021)
- [Amnesiac Machine Learning](https://mlsanity.com/paper/2010.10981/) (2020)
- [Federated Optimization:Distributed Optimization Beyond the Datacenter](https://mlsanity.com/paper/1511.03575/) (2015)
- [Boundary Unlearning](https://mlsanity.com/paper/2303.11570/) (2023)
- [The Right to be Forgotten in Federated Learning: An Efficient Realization with Rapid Retraining](https://mlsanity.com/paper/2203.07320/) (2022)
- [Federated Unlearning via Class-Discriminative Pruning](https://mlsanity.com/paper/2110.11794/) (2021)
- [Decentralized Federated Averaging](https://mlsanity.com/paper/2104.11375/) (2021)
- [Distributionally Robust Federated Averaging](https://mlsanity.com/paper/2102.12660/) (2021)
- [Federated Unlearning: How to Efficiently Erase a Client in FL?](https://mlsanity.com/paper/2207.05521/) (2022)
- [Federated Unlearning with Knowledge Distillation](https://mlsanity.com/paper/2201.09441/) (2022)
- [A Survey on Federated Unlearning: Challenges, Methods, and Future Directions](https://mlsanity.com/paper/2310.20448/) (2023)
- [VeriFi: Towards Verifiable Federated Unlearning](https://mlsanity.com/paper/2205.12709/) (2022)
- [Verifiable and Provably Secure Machine Unlearning](https://mlsanity.com/paper/2210.09126/) (2022)
- [An Introduction to Machine Unlearning](https://mlsanity.com/paper/2209.00939/) (2022)
- [A unified PAC-Bayesian framework for machine unlearning via information risk minimization](https://mlsanity.com/paper/2106.00265/) (2021)

## Built on by (most cited first, 14 of 14)

- [Ten Challenging Problems in Federated Foundation Models](https://mlsanity.com/paper/2502.12176/) (2025)
- [Unlearning through Knowledge Overwriting: Reversible Federated Unlearning via Selective Sparse Adapter](https://mlsanity.com/paper/2502.20709/) (2025)
- [Towards Privacy-Guaranteed Label Unlearning in Vertical Federated Learning: Few-Shot Forgetting without Disclosure](https://mlsanity.com/paper/2410.10922/) (2024)
- [ErrorEraser: Unlearning Data Bias for Improved Continual Learning](https://mlsanity.com/paper/2506.09347/) (2025)
- [FedQUIT: On-Device Federated Unlearning via a Quasi-Competent Virtual Teacher](https://mlsanity.com/paper/2408.07587/) (2024)
- [Exploring Incremental Unlearning: Techniques, Challenges, and Future Directions](https://mlsanity.com/paper/2502.16708/) (2025)
- [Federated Unlearning Made Practical: Seamless Integration via Negated Pseudo-Gradients](https://mlsanity.com/paper/2504.05822/) (2025)
- [ToFU: Transforming How Federated Learning Systems Forget User Data](https://mlsanity.com/paper/2509.15861/) (2025)
- [Forgetting Through Transforming: Enabling Federated Unlearning via Class-Aware Representation Transformation](https://mlsanity.com/paper/2410.06848/) (2024)
- [Sky of Unlearning (SoUL): Rewiring Federated Machine Unlearning via Selective Pruning](https://mlsanity.com/paper/2504.01705/) (2025)
- [Lethe: Principled Dual-Stream Update for Persistent Knowledge Erasure in Federated Unlearning](https://mlsanity.com/paper/2601.22601/) (2026)
- [f-FUM: Federated Unlearning via min--max and f-divergence](https://mlsanity.com/paper/2602.06187/) (2026)
- [Asynchronous Federated Unlearning with Invariance Calibration for Medical Imaging](https://mlsanity.com/paper/2604.26809/) (2026)
- [pFedUL: Layer-Aware Federated Unlearning for Personalized Federated Learning](https://mlsanity.com/paper/2606.16304/) (2026)

## Abstract (arXiv)

In recent years, Federated Learning (FL) has garnered significant attention as a distributed machine learning paradigm. To facilitate the implementation of the right to be forgotten, the concept of federated machine unlearning (FMU) has also emerged. However, current FMU approaches often involve additional time-consuming steps and may not offer comprehensive unlearning capabilities, which renders them less practical in real FL scenarios. In this paper, we introduce FedAU, an innovative and efficient FMU framework aimed at overcoming these limitations. Specifically, FedAU incorporates a lightweight auxiliary unlearning module into the learning process and employs a straightforward linear operation to facilitate unlearning. This approach eliminates the requirement for extra time-consuming steps, rendering it well-suited for FL. Furthermore, FedAU exhibits remarkable versatility. It not only enables multiple clients to carry out unlearning tasks concurrently but also supports unlearning at various levels of granularity, including individual data samples, specific classes, and even at the client level. We conducted extensive experiments on MNIST, CIFAR10, and CIFAR100 datasets to evaluate the performance of FedAU. The results demonstrate that FedAU effectively achieves the desired unlearning effect while maintaining model accuracy. Our code is availiable at https://github.com/Liar-Mask/FedAU.