# Federated Unlearning with Gradient Descent and Conflict Mitigation

arXiv: https://arxiv.org/abs/2412.20200 · HTML: https://mlsanity.com/paper/2412.20200/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Placed in Federated unlearning and poisoning, one of 12 directions in LLM safety and security.
- Builds on 6 of its 24 references that are in the map.
- First posted on arXiv 28 Dec 2024.

## Placed in

- LLM safety and security / [Federated unlearning and poisoning](https://mlsanity.com/safety/federated-unlearning-and-poisoning/)

## Builds on (most cited first, 6 of 6)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Gradient Projection Memory for Continual Learning](https://mlsanity.com/paper/2103.09762/) (2021)
- [Federated Unlearning: How to Efficiently Erase a Client in FL?](https://mlsanity.com/paper/2207.05521/) (2022)
- [Characterizing signal propagation to close the performance gap in unnormalized ResNets](https://mlsanity.com/paper/2101.08692/) (2021)
- [Federated Learning with Fair Averaging](https://mlsanity.com/paper/2104.14937/) (2021)

## Built on by (most cited first, 17 of 17)

- [Learning to Unlearn while Retaining: Combating Gradient Conflicts in Machine Unlearning](https://mlsanity.com/paper/2503.06339/) (2025)
- [NoT: Federated Unlearning via Weight Negation](https://mlsanity.com/paper/2503.05657/) (2025)
- [EASE: Federated Multimodal Unlearning via Entanglement-Aware Anchor Closure](https://mlsanity.com/paper/2605.00733/) (2026)
- [Distribution Preference Optimization: A Fine-grained Perspective for LLM Unlearning](https://mlsanity.com/paper/2510.04773/) (2025)
- [FedCARE: Federated Unlearning with Conflict-Aware Projection and Relearning-Resistant Recovery](https://mlsanity.com/paper/2601.22589/) (2026)
- [EFU: Enforcing Federated Unlearning via Functional Encryption](https://mlsanity.com/paper/2508.07873/) (2025)
- [REMISVFU: Vertical Federated Unlearning via Representation Misdirection for Intermediate Output Feature](https://mlsanity.com/paper/2512.10348/) (2025)
- [Rethinking Federated Unlearning via the Lens of Memorization](https://mlsanity.com/paper/2605.24545/) (2026)
- [FedSGT: Exact Federated Unlearning via Sequential Group-based Training](https://mlsanity.com/paper/2511.23393/) (2025)
- [Lethe: Principled Dual-Stream Update for Persistent Knowledge Erasure in Federated Unlearning](https://mlsanity.com/paper/2601.22601/) (2026)
- [Computation and Communication Efficient Federated Unlearning via On-server Gradient Conflict Mitigation and Expression](https://mlsanity.com/paper/2603.13795/) (2026)
- [Client-Verifiable and Efficient Federated Unlearning in Low-Altitude Wireless Networks](https://mlsanity.com/paper/2603.29688/) (2026)
- [Asynchronous Federated Unlearning with Invariance Calibration for Medical Imaging](https://mlsanity.com/paper/2604.26809/) (2026)
- [pFedUL: Layer-Aware Federated Unlearning for Personalized Federated Learning](https://mlsanity.com/paper/2606.16304/) (2026)
- [Priority-Aware Learning-Unlearning Correction for Dynamic Decentralized LoRA Fine-Tuning](https://mlsanity.com/paper/2606.22878/) (2026)
- [FedUP: One-Shot Federated Unlearning via Centroid-Guided Plug-in Filters](https://mlsanity.com/paper/2606.24113/) (2026)
- [MMFGU: Multimodal Federated Graph Unlearning](https://mlsanity.com/paper/2607.28708/) (2026)

## Abstract (arXiv)

Federated Learning (FL) has received much attention in recent years. However, although clients are not required to share their data in FL, the global model itself can implicitly remember clients' local data. Therefore, it's necessary to effectively remove the target client's data from the FL global model to ease the risk of privacy leakage and implement ``the right to be forgotten". Federated Unlearning (FU) has been considered a promising way to remove data without full retraining. But the model utility easily suffers significant reduction during unlearning due to the gradient conflicts. Furthermore, when conducting the post-training to recover the model utility, the model is prone to move back and revert what has already been unlearned. To address these issues, we propose Federated Unlearning with Orthogonal Steepest Descent (FedOSD). We first design an unlearning Cross-Entropy loss to overcome the convergence issue of the gradient ascent. A steepest descent direction for unlearning is then calculated in the condition of being non-conflicting with other clients' gradients and closest to the target client's gradient. This benefits to efficiently unlearn and mitigate the model utility reduction. After unlearning, we recover the model utility by maintaining the achievement of unlearning. Finally, extensive experiments in several FL scenarios verify that FedOSD outperforms the SOTA FU algorithms in terms of unlearning and model utility.