# Subspace based Federated Unlearning

arXiv: https://arxiv.org/abs/2302.12448 · HTML: https://mlsanity.com/paper/2302.12448/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Placed in Federated unlearning and poisoning, one of 12 directions in LLM safety and security.
- Builds on 38 of its 58 references that are in the map.
- First posted on arXiv 24 Feb 2023.

## Placed in

- LLM safety and security / [Federated unlearning and poisoning](https://mlsanity.com/safety/federated-unlearning-and-poisoning/)

## Builds on (most cited first, 30 of 38)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Federated Learning: Challenges, Methods, and Future Directions](https://mlsanity.com/paper/1908.07873/) (2019)
- [On the Convergence of FedAvg on Non-IID Data](https://mlsanity.com/paper/1907.02189/) (2019)
- [Federated Learning with Non-IID Data](https://mlsanity.com/paper/1806.00582/) (2018)
- [BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain](https://mlsanity.com/paper/1708.06733/) (2017)
- [Machine Unlearning](https://mlsanity.com/paper/1912.03817/) (2019)
- [Towards Federated Learning at Scale: System Design](https://mlsanity.com/paper/1902.01046/) (2019)
- [Model-Contrastive Federated Learning](https://mlsanity.com/paper/2103.16257/) (2021)
- [How To Backdoor Federated Learning](https://mlsanity.com/paper/1807.00459/) (2018)
- [Exploiting Unintended Feature Leakage in Collaborative Learning](https://mlsanity.com/paper/1805.04049/) (2018)
- [Making AI Forget You: Data Deletion in Machine Learning](https://mlsanity.com/paper/1907.05012/) (2019)
- [A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protection](https://mlsanity.com/paper/1907.09693/) (2019)
- [Remember What You Want to Forget: Algorithms for Machine Unlearning](https://mlsanity.com/paper/2103.03279/) (2021)
- [Orthogonal Gradient Descent for Continual Learning](https://mlsanity.com/paper/1910.07104/) (2019)
- [Gradient Projection Memory for Continual Learning](https://mlsanity.com/paper/2103.09762/) (2021)
- [Amnesiac Machine Learning](https://mlsanity.com/paper/2010.10981/) (2020)
- [Unrolling SGD: Understanding Factors Influencing Machine Unlearning](https://mlsanity.com/paper/2109.13398/) (2021)
- [Approximate Data Deletion from Machine Learning Models](https://mlsanity.com/paper/2002.10077/) (2020)
- [A Hybrid Approach to Privacy-Preserving Federated Learning](https://mlsanity.com/paper/1812.03224/) (2018)
- [DeltaGrad: Rapid retraining of machine learning models](https://mlsanity.com/paper/2006.14755/) (2020)
- [Federated Unlearning via Class-Discriminative Pruning](https://mlsanity.com/paper/2110.11794/) (2021)
- [Machine Unlearning: Linear Filtration for Logit-based Classifiers](https://mlsanity.com/paper/2002.02730/) (2020)
- [Federated Unlearning: How to Efficiently Erase a Client in FL?](https://mlsanity.com/paper/2207.05521/) (2022)
- [Bypassing the Ambient Dimension: Private SGD with Gradient Subspace Identification](https://mlsanity.com/paper/2007.03813/) (2020)
- [Comprehensive Privacy Analysis of Deep Learning: Passive and Active White-box Inference Attacks against Centralized and Federated Learning](https://mlsanity.com/paper/1812.00910/) (2018)
- [Federated Unlearning with Knowledge Distillation](https://mlsanity.com/paper/2201.09441/) (2022)

## Built on by (most cited first, 17 of 17)

- [A Survey on Federated Unlearning: Challenges, Methods, and Future Directions](https://mlsanity.com/paper/2310.20448/) (2023)
- [Exploring the Landscape of Machine Unlearning: A Comprehensive Survey and Taxonomy](https://mlsanity.com/paper/2305.06360/) (2023)
- [Federated Unlearning: A Survey on Methods, Design Guidelines, and Evaluation Metrics](https://mlsanity.com/paper/2401.05146/) (2024)
- [Deep Unlearning: Fast and Efficient Gradient-free Approach to Class Forgetting](https://mlsanity.com/paper/2312.00761/) (2023)
- [Federated Orthogonal Training: Mitigating Global Catastrophic Forgetting in Continual Federated Learning](https://mlsanity.com/paper/2309.01289/) (2023)
- [Ferrari: Federated Feature Unlearning via Optimizing Feature Sensitivity](https://mlsanity.com/paper/2405.17462/) (2024)
- [Towards Privacy-Guaranteed Label Unlearning in Vertical Federated Learning: Few-Shot Forgetting without Disclosure](https://mlsanity.com/paper/2410.10922/) (2024)
- [Split Unlearning](https://mlsanity.com/paper/2308.10422/) (2023)
- [Efficient Federated Unlearning with Adaptive Differential Privacy Preservation](https://mlsanity.com/paper/2411.11044/) (2024)
- [Ready2Unlearn: A Learning-Time Approach for Preparing Models with Future Unlearning Readiness](https://mlsanity.com/paper/2505.10845/) (2025)
- [A Survey on Federated Unlearning: Challenges and Opportunities](https://mlsanity.com/paper/2403.02437/) (2024)
- [Convergent Differential Privacy Analysis for General Federated Learning](https://mlsanity.com/paper/2408.15621/) (2024)
- [Streamlined Federated Unlearning: Unite as One to Be Highly Efficient](https://mlsanity.com/paper/2412.00126/) (2024)
- [FedSGT: Exact Federated Unlearning via Sequential Group-based Training](https://mlsanity.com/paper/2511.23393/) (2025)
- [Machine Unlearning in Low-Dimensional Feature Subspace](https://mlsanity.com/paper/2601.22456/) (2026)
- [f-FUM: Federated Unlearning via min--max and f-divergence](https://mlsanity.com/paper/2602.06187/) (2026)
- [Client-Verifiable and Efficient Federated Unlearning in Low-Altitude Wireless Networks](https://mlsanity.com/paper/2603.29688/) (2026)

## Abstract (arXiv)

Federated learning (FL) enables multiple clients to train a machine learning model collaboratively without exchanging their local data. Federated unlearning is an inverse FL process that aims to remove a specified target client's contribution in FL to satisfy the user's right to be forgotten. Most existing federated unlearning algorithms require the server to store the history of the parameter updates, which is not applicable in scenarios where the server storage resource is constrained. In this paper, we propose a simple-yet-effective subspace based federated unlearning method, dubbed SFU, that lets the global model perform gradient ascent in the orthogonal space of input gradient spaces formed by other clients to eliminate the target client's contribution without requiring additional storage. Specifically, the server first collects the gradients generated from the target client after performing gradient ascent, and the input representation matrix is computed locally by the remaining clients. We also design a differential privacy method to protect the privacy of the representation matrix. Then the server merges those representation matrices to get the input gradient subspace and updates the global model in the orthogonal subspace of the input gradient subspace to complete the forgetting task with minimal model performance degradation. Experiments on MNIST, CIFAR10, and CIFAR100 show that SFU outperforms several state-of-the-art (SOTA) federated unlearning algorithms by a large margin in various settings.