# Fast Machine Unlearning Without Retraining Through Selective Synaptic Dampening

arXiv: https://arxiv.org/abs/2308.07707 · HTML: https://mlsanity.com/paper/2308.07707/

## Key facts

- Cited by 181 papers in the map (data through 14 Oct 2026).
- Placed in Machine unlearning, one of 12 directions in LLM safety and security.
- Builds on 26 of its 38 references that are in the map.
- First posted on arXiv 15 Aug 2023.

## Placed in

- LLM safety and security / [Machine unlearning](https://mlsanity.com/safety/machine-unlearning/)

## Builds on (most cited first, 26 of 26)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/1610.05820/) (2016)
- [The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks](https://mlsanity.com/paper/1802.08232/) (2018)
- [Eternal Sunshine of the Spotless Net: Selective Forgetting in Deep Networks](https://mlsanity.com/paper/1911.04933/) (2019)
- [Certified Data Removal from Machine Learning Models](https://mlsanity.com/paper/1911.03030/) (2019)
- [Making AI Forget You: Data Deletion in Machine Learning](https://mlsanity.com/paper/1907.05012/) (2019)
- [Does Learning Require Memorization? A Short Tale about a Long Tail](https://mlsanity.com/paper/1906.05271/) (2019)
- [Remember What You Want to Forget: Algorithms for Machine Unlearning](https://mlsanity.com/paper/2103.03279/) (2021)
- [Membership Inference Attacks on Machine Learning: A Survey](https://mlsanity.com/paper/2103.07853/) (2021)
- [Amnesiac Machine Learning](https://mlsanity.com/paper/2010.10981/) (2020)
- [A Survey of Machine Unlearning](https://mlsanity.com/paper/2209.02299/) (2022)
- [Fast Yet Effective Machine Unlearning](https://mlsanity.com/paper/2111.08947/) (2021)
- [Can Bad Teaching Induce Forgetting? Unlearning in Deep Networks using an Incompetent Teacher](https://mlsanity.com/paper/2205.08096/) (2022)
- [Forgetting Outside the Box: Scrubbing Deep Networks of Information Accessible from Input-Output Observations](https://mlsanity.com/paper/2003.02960/) (2020)
- [Mixed-Privacy Forgetting in Deep Networks](https://mlsanity.com/paper/2012.13431/) (2020)
- [Zero-Shot Machine Unlearning](https://mlsanity.com/paper/2201.05629/) (2022)
- [Variational Bayesian Unlearning](https://mlsanity.com/paper/2010.12883/) (2020)
- [Continuous Learning in Single-Incremental-Task Scenarios](https://mlsanity.com/paper/1806.08568/) (2018)
- [Deep Unlearning via Randomized Conditionally Independent Hessians](https://mlsanity.com/paper/2204.07655/) (2022)
- [On the geometry of generalization and memorization in deep neural networks](https://mlsanity.com/paper/2105.14602/) (2021)
- [Deep Regression Unlearning](https://mlsanity.com/paper/2210.08196/) (2022)
- [Elastic Weight Consolidation (EWC): Nuts and Bolts](https://mlsanity.com/paper/2105.04093/) (2021)

## Built on by (most cited first, 30 of 181)

- [The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearning](https://mlsanity.com/paper/2403.03218/) (2024)
- [In-Context Unlearning: Language Models as Few Shot Unlearners](https://mlsanity.com/paper/2310.07579/) (2023)
- [Guardrail Baselines for Unlearning in LLMs](https://mlsanity.com/paper/2403.03329/) (2024)
- [Scissorhands: Scrub Data Influence via Connection Sensitivity in Networks](https://mlsanity.com/paper/2401.06187/) (2024)
- [Unified Gradient-Based Machine Unlearning with Remain Geometry Enhancement](https://mlsanity.com/paper/2409.19732/) (2024)
- [Dissecting Language Models: Machine Unlearning via Selective Pruning](https://mlsanity.com/paper/2403.01267/) (2024)
- [Extending Activation Steering to Broad Skills and Multiple Behaviours](https://mlsanity.com/paper/2403.05767/) (2024)
- [Machine Unlearning Fails to Remove Data Poisoning Attacks](https://mlsanity.com/paper/2406.17216/) (2024)
- [Erasing Conceptual Knowledge from Language Models](https://mlsanity.com/paper/2410.02760/) (2024)
- [On Effects of Steering Latent Representation for Large Language Model Unlearning](https://mlsanity.com/paper/2408.06223/) (2024)
- [Corrective Machine Unlearning](https://mlsanity.com/paper/2402.14015/) (2024)
- [Decoupled Distillation to Erase: A General Unlearning Method for Any Class-centric Tasks](https://mlsanity.com/paper/2503.23751/) (2025)
- [Deep Unlearning: Fast and Efficient Gradient-free Approach to Class Forgetting](https://mlsanity.com/paper/2312.00761/) (2023)
- [Pruning for Robust Concept Erasing in Diffusion Models](https://mlsanity.com/paper/2405.16534/) (2024)
- [Data Attribution for Text-to-Image Models by Unlearning Synthesized Images](https://mlsanity.com/paper/2406.09408/) (2024)
- [SAEs Can Improve Unlearning: Dynamic Sparse Autoencoder Guardrails for Precision Unlearning in LLMs](https://mlsanity.com/paper/2504.08192/) (2025)
- [The Utility and Complexity of in- and out-of-Distribution Machine Unlearning](https://mlsanity.com/paper/2412.09119/) (2024)
- [Learning to Unlearn for Robust Machine Unlearning](https://mlsanity.com/paper/2407.10494/) (2024)
- [Targeted Unlearning with Single Layer Unlearning Gradient](https://mlsanity.com/paper/2407.11867/) (2024)
- [MU-Bench: A Multitask Multimodal Benchmark for Machine Unlearning](https://mlsanity.com/paper/2406.14796/) (2024)
- [MUNBa: Machine Unlearning via Nash Bargaining](https://mlsanity.com/paper/2411.15537/) (2024)
- [ConeSep: Cone-based Robust Noise-Unlearning Compositional Network for Composed Image Retrieval](https://mlsanity.com/paper/2604.20358/) (2026)
- [An Information Theoretic Approach to Machine Unlearning](https://mlsanity.com/paper/2402.01401/) (2024)
- [A Survey on Machine Unlearning: Techniques and New Emerged Privacy Risks](https://mlsanity.com/paper/2406.06186/) (2024)
- [Boosting Alignment for Post-Unlearning Text-to-Image Generative Models](https://mlsanity.com/paper/2412.07808/) (2024)
- [NoT: Federated Unlearning via Weight Negation](https://mlsanity.com/paper/2503.05657/) (2025)
- [Potion: Towards Poison Unlearning](https://mlsanity.com/paper/2406.09173/) (2024)
- [DUCK: Distance-based Unlearning via Centroid Kinematics](https://mlsanity.com/paper/2312.02052/) (2023)
- [Remaining-data-free Machine Unlearning by Suppressing Sample Contribution](https://mlsanity.com/paper/2402.15109/) (2024)
- [Loss-Free Machine Unlearning](https://mlsanity.com/paper/2402.19308/) (2024)

## Abstract (arXiv)

Machine unlearning, the ability for a machine learning model to forget, is becoming increasingly important to comply with data privacy regulations, as well as to remove harmful, manipulated, or outdated information. The key challenge lies in forgetting specific information while protecting model performance on the remaining data. While current state-of-the-art methods perform well, they typically require some level of retraining over the retained data, in order to protect or restore model performance. This adds computational overhead and mandates that the training data remain available and accessible, which may not be feasible. In contrast, other methods employ a retrain-free paradigm, however, these approaches are prohibitively computationally expensive and do not perform on par with their retrain-based counterparts. We present Selective Synaptic Dampening (SSD), a novel two-step, post hoc, retrain-free approach to machine unlearning which is fast, performant, and does not require long-term storage of the training data. First, SSD uses the Fisher information matrix of the training and forgetting data to select parameters that are disproportionately important to the forget set. Second, SSD induces forgetting by dampening these parameters proportional to their relative importance to the forget set with respect to the wider training data. We evaluate our method against several existing unlearning methods in a range of experiments using ResNet18 and Vision Transformer. Results show that the performance of SSD is competitive with retrain-based post hoc methods, demonstrating the viability of retrain-free post hoc unlearning approaches.