# Loss-Free Machine Unlearning

arXiv: https://arxiv.org/abs/2402.19308 · HTML: https://mlsanity.com/paper/2402.19308/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Placed in Machine unlearning, one of 12 directions in LLM safety and security.
- Builds on 8 of its 10 references that are in the map.
- First posted on arXiv 29 Feb 2024.

## Placed in

- LLM safety and security / [Machine unlearning](https://mlsanity.com/safety/machine-unlearning/)

## Builds on (most cited first, 8 of 8)

- [Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/1610.05820/) (2016)
- [Memory Aware Synapses: Learning what (not) to forget](https://mlsanity.com/paper/1711.09601/) (2017)
- [Eternal Sunshine of the Spotless Net: Selective Forgetting in Deep Networks](https://mlsanity.com/paper/1911.04933/) (2019)
- [Amnesiac Machine Learning](https://mlsanity.com/paper/2010.10981/) (2020)
- [Towards Unbounded Machine Unlearning](https://mlsanity.com/paper/2302.09880/) (2023)
- [Can Bad Teaching Induce Forgetting? Unlearning in Deep Networks using an Incompetent Teacher](https://mlsanity.com/paper/2205.08096/) (2022)
- [Fast Machine Unlearning Without Retraining Through Selective Synaptic Dampening](https://mlsanity.com/paper/2308.07707/) (2023)
- [Zero-Shot Machine Unlearning](https://mlsanity.com/paper/2201.05629/) (2022)

## Built on by (most cited first, 13 of 13)

- [Potion: Towards Poison Unlearning](https://mlsanity.com/paper/2406.09173/) (2024)
- [RAZOR: Ratio-Aware Layer Editing for Targeted Unlearning in Vision Transformers and Diffusion Models](https://mlsanity.com/paper/2603.14819/) (2026)
- [Forget Vectors at Play: Universal Input Perturbations Driving Machine Unlearning in Image Classification](https://mlsanity.com/paper/2412.16780/) (2024)
- [Exploring Incremental Unlearning: Techniques, Challenges, and Future Directions](https://mlsanity.com/paper/2502.16708/) (2025)
- [One-Point Contraction: Erasing Representational Separability toward Irreversible Deep Forgetting](https://mlsanity.com/paper/2507.07754/) (2025)
- [MUC: Machine Unlearning for Contrastive Learning with Black-box Evaluation](https://mlsanity.com/paper/2406.03603/) (2024)
- [IMU: Influence-guided Machine Unlearning](https://mlsanity.com/paper/2508.01620/) (2025)
- [On the importance of multiple training seeds for evaluating machine unlearning](https://mlsanity.com/paper/2510.26714/) (2025)
- [Efficient Unlearning through Maximizing Relearning Convergence Delay](https://mlsanity.com/paper/2604.09391/) (2026)
- [SUPREME: A Multi-GPU Framework for Reproducible Image Unlearning Method Evaluation](https://mlsanity.com/paper/2606.00380/) (2026)
- [DFMU: Data-Frugal Machine Unlearning](https://mlsanity.com/paper/2606.25410/) (2026)
- [Repurposing Obsolete Representations for Post-Deployment Adaptation](https://mlsanity.com/paper/2610.01453/) (2026)
- [UnAct: Gradient-Free Unlearning via Targeted Activation Intervention](https://mlsanity.com/paper/2610.04426/) (2026)

## Abstract (arXiv)

We present a machine unlearning approach that is both retraining- and label-free. Most existing machine unlearning approaches require a model to be fine-tuned to remove information while preserving performance. This is computationally expensive and necessitates the storage of the whole dataset for the lifetime of the model. Retraining-free approaches often utilise Fisher information, which is derived from the loss and requires labelled data which may not be available. Thus, we present an extension to the Selective Synaptic Dampening algorithm, substituting the diagonal of the Fisher information matrix for the gradient of the l2 norm of the model output to approximate sensitivity. We evaluate our method in a range of experiments using ResNet18 and Vision Transformer. Results show our label-free method is competitive with existing state-of-the-art approaches.