# Unlearning through Knowledge Overwriting: Reversible Federated Unlearning via Selective Sparse Adapter

arXiv: https://arxiv.org/abs/2502.20709 · HTML: https://mlsanity.com/paper/2502.20709/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Placed in Federated unlearning and poisoning, one of 12 directions in LLM safety and security.
- Builds on 24 of its 40 references that are in the map.
- First posted on arXiv 28 Feb 2025.

## Placed in

- LLM safety and security / [Federated unlearning and poisoning](https://mlsanity.com/safety/federated-unlearning-and-poisoning/)

## Builds on (most cited first, 24 of 24)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Machine Unlearning](https://mlsanity.com/paper/1912.03817/) (2019)
- [Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descent](https://mlsanity.com/paper/1902.06720/) (2019)
- [Eternal Sunshine of the Spotless Net: Selective Forgetting in Deep Networks](https://mlsanity.com/paper/1911.04933/) (2019)
- [Certified Data Removal from Machine Learning Models](https://mlsanity.com/paper/1911.03030/) (2019)
- [Approximate Data Deletion from Machine Learning Models](https://mlsanity.com/paper/2002.10077/) (2020)
- [Fast Yet Effective Machine Unlearning](https://mlsanity.com/paper/2111.08947/) (2021)
- [Graph Unlearning](https://mlsanity.com/paper/2103.14991/) (2021)
- [Machine Unlearning for Random Forests](https://mlsanity.com/paper/2009.05567/) (2020)
- [The Right to be Forgotten in Federated Learning: An Efficient Realization with Rapid Retraining](https://mlsanity.com/paper/2203.07320/) (2022)
- [Federated Unlearning via Class-Discriminative Pruning](https://mlsanity.com/paper/2110.11794/) (2021)
- [Machine Unlearning: Linear Filtration for Logit-based Classifiers](https://mlsanity.com/paper/2002.02730/) (2020)
- [Federated Unlearning: How to Efficiently Erase a Client in FL?](https://mlsanity.com/paper/2207.05521/) (2022)
- [DMCP: Differentiable Markov Channel Pruning for Neural Networks](https://mlsanity.com/paper/2005.03354/) (2020)
- [FedRecover: Recovering from Poisoning Attacks in Federated Learning using Historical Information](https://mlsanity.com/paper/2210.10936/) (2022)
- [Federated Continual Learning via Knowledge Fusion: A Survey](https://mlsanity.com/paper/2312.16475/) (2023)
- [Formalizing Data Deletion in the Context of the Right to be Forgotten](https://mlsanity.com/paper/2002.10635/) (2020)
- [A Review on Machine Unlearning](https://mlsanity.com/paper/2411.11315/) (2024)
- [Federated Unlearning](https://mlsanity.com/paper/2012.13891/) (2020)
- [Towards Scalable Exact Machine Unlearning Using Parameter-Efficient Fine-Tuning](https://mlsanity.com/paper/2406.16257/) (2024)
- [Unlearning during Learning: An Efficient Federated Machine Unlearning Method](https://mlsanity.com/paper/2405.15474/) (2024)
- [Class Clown: Data Redaction in Machine Unlearning at Enterprise Scale](https://mlsanity.com/paper/2012.04699/) (2020)
- [Fast-NTK: Parameter-Efficient Unlearning for Large-Scale Models](https://mlsanity.com/paper/2312.14923/) (2023)

## Built on by (most cited first, 16 of 16)

- [Class-wise Balancing Data Replay for Federated Class-Incremental Learning](https://mlsanity.com/paper/2507.07712/) (2025)
- [EASE: Federated Multimodal Unlearning via Entanglement-Aware Anchor Closure](https://mlsanity.com/paper/2605.00733/) (2026)
- [Certified Unlearning in Decentralized Federated Learning](https://mlsanity.com/paper/2601.06436/) (2026)
- [EFU: Enforcing Federated Unlearning via Functional Encryption](https://mlsanity.com/paper/2508.07873/) (2025)
- [From Privacy to Trust in the Agentic Era: A Taxonomy of Challenges in Trustworthy Federated Learning Through the Lens of Trust Report 2.0](https://mlsanity.com/paper/2507.15796/) (2025)
- [Federated Graph Unlearning](https://mlsanity.com/paper/2508.02485/) (2025)
- [Towards Verifiable Federated Unlearning: Framework, Challenges, and The Road Ahead](https://mlsanity.com/paper/2510.00833/) (2025)
- [Gains: Fine-grained Federated Domain Adaptation in Open Set](https://mlsanity.com/paper/2510.15967/) (2025)
- [Computation and Communication Efficient Federated Unlearning via On-server Gradient Conflict Mitigation and Expression](https://mlsanity.com/paper/2603.13795/) (2026)
- [oslash Source Models Leak What They Shouldn't nrightarrow: Unlearning Zero-Shot Transfer in Domain Adaptation Through Adversarial Optimization](https://mlsanity.com/paper/2604.08238/) (2026)
- [SoK: Unlearnability and Unlearning for Model Dememorization](https://mlsanity.com/paper/2605.11592/) (2026)
- [Image Feature Fusion-based Federated Client Unlearning (FCU)](https://mlsanity.com/paper/2605.26715/) (2026)
- [pFedUL: Layer-Aware Federated Unlearning for Personalized Federated Learning](https://mlsanity.com/paper/2606.16304/) (2026)
- [FedUP: One-Shot Federated Unlearning via Centroid-Guided Plug-in Filters](https://mlsanity.com/paper/2606.24113/) (2026)
- [IREU: Identity-Related Encoder-Only Unlearning for Customized Portrait Generation](https://mlsanity.com/paper/2606.29880/) (2026)
- [Lethe: How Hard Is It to Forget? A Benchmark for Federated Unlearning in Medical Imaging](https://mlsanity.com/paper/2608.01094/) (2026)

## Abstract (arXiv)

Federated Learning is a promising paradigm for privacy-preserving collaborative model training. In practice, it is essential not only to continuously train the model to acquire new knowledge but also to guarantee old knowledge the right to be forgotten (i.e., federated unlearning), especially for privacy-sensitive information or harmful knowledge. However, current federated unlearning methods face several challenges, including indiscriminate unlearning of cross-client knowledge, irreversibility of unlearning, and significant unlearning costs. To this end, we propose a method named FUSED, which first identifies critical layers by analyzing each layer's sensitivity to knowledge and constructs sparse unlearning adapters for sensitive ones. Then, the adapters are trained without altering the original parameters, overwriting the unlearning knowledge with the remaining knowledge. This knowledge overwriting process enables FUSED to mitigate the effects of indiscriminate unlearning. Moreover, the introduction of independent adapters makes unlearning reversible and significantly reduces the unlearning costs. Finally, extensive experiments on three datasets across various unlearning scenarios demonstrate that FUSED's effectiveness is comparable to Retraining, surpassing all other baselines while greatly reducing unlearning costs.