# Certified Unlearning for Neural Networks

arXiv: https://arxiv.org/abs/2506.06985 · HTML: https://mlsanity.com/paper/2506.06985/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Placed in Machine unlearning, one of 12 directions in LLM safety and security.
- Builds on 26 of its 34 references that are in the map.
- First posted on arXiv 8 Jun 2025.

## Placed in

- LLM safety and security / [Machine unlearning](https://mlsanity.com/safety/machine-unlearning/)

## Builds on (most cited first, 26 of 26)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Deep Learning with Differential Privacy](https://mlsanity.com/paper/1607.00133/) (2016)
- [Machine Unlearning](https://mlsanity.com/paper/1912.03817/) (2019)
- [An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks](https://mlsanity.com/paper/1312.6211/) (2013)
- [Certified Data Removal from Machine Learning Models](https://mlsanity.com/paper/1911.03030/) (2019)
- [Making AI Forget You: Data Deletion in Machine Learning](https://mlsanity.com/paper/1907.05012/) (2019)
- [Who's Harry Potter? Approximate Unlearning in LLMs](https://mlsanity.com/paper/2310.02238/) (2023)
- [Remember What You Want to Forget: Algorithms for Machine Unlearning](https://mlsanity.com/paper/2103.03279/) (2021)
- [Amnesiac Machine Learning](https://mlsanity.com/paper/2010.10981/) (2020)
- [A Survey of Machine Unlearning](https://mlsanity.com/paper/2209.02299/) (2022)
- [Descent-to-Delete: Gradient-Based Methods for Machine Unlearning](https://mlsanity.com/paper/2007.02923/) (2020)
- [Towards Unbounded Machine Unlearning](https://mlsanity.com/paper/2302.09880/) (2023)
- [Rethinking Machine Unlearning for Large Language Models](https://mlsanity.com/paper/2402.08787/) (2024)
- [Super-Convergence: Very Fast Training of Neural Networks Using Large Learning Rates](https://mlsanity.com/paper/1708.07120/) (2017)
- [On the Necessity of Auditable Algorithmic Definitions for Machine Unlearning](https://mlsanity.com/paper/2110.11891/) (2021)
- [Mixed-Privacy Forgetting in Deep Networks](https://mlsanity.com/paper/2012.13431/) (2020)
- [Privacy Amplification by Iteration](https://mlsanity.com/paper/1808.06651/) (2018)
- [Hypothesis Testing Interpretations and Renyi Differential Privacy](https://mlsanity.com/paper/1905.09982/) (2019)
- [Forget Unlearning: Towards True Data-Deletion in Machine Learning](https://mlsanity.com/paper/2210.08911/) (2022)
- [Are we making progress in unlearning? Findings from the first NeurIPS unlearning competition](https://mlsanity.com/paper/2406.09073/) (2024)
- [Langevin Unlearning: A New Perspective of Noisy Gradient Descent for Machine Unlearning](https://mlsanity.com/paper/2401.10371/) (2024)
- [Towards Certified Unlearning for Deep Neural Networks](https://mlsanity.com/paper/2408.00920/) (2024)
- [Certified Minimax Unlearning with Generalization Rates and Deletion Capacity](https://mlsanity.com/paper/2312.10336/) (2023)
- [Privacy Amplification by Mixing and Diffusion Mechanisms](https://mlsanity.com/paper/1905.12264/) (2019)
- [The Utility and Complexity of in- and out-of-Distribution Machine Unlearning](https://mlsanity.com/paper/2412.09119/) (2024)
- [Rewind-to-Delete: Certified Machine Unlearning for Nonconvex Functions](https://mlsanity.com/paper/2409.09778/) (2024)

## Built on by (most cited first, 27 of 27)

- [AutoSOTA: An End-to-End Automated Research System for State-of-the-Art AI Model Discovery](https://mlsanity.com/paper/2604.05550/) (2026)
- [On the Impossibility of Retrain Equivalence in Machine Unlearning](https://mlsanity.com/paper/2510.16629/) (2025)
- [Distributional Machine Unlearning via Selective Data Removal](https://mlsanity.com/paper/2507.15112/) (2025)
- [OFMU: Optimization-Driven Framework for Machine Unlearning](https://mlsanity.com/paper/2509.22483/) (2025)
- [Machine Unlearning in the Era of Quantum Machine Learning: An Empirical Study](https://mlsanity.com/paper/2512.19253/) (2025)
- [How to sketch a learning algorithm](https://mlsanity.com/paper/2604.07328/) (2026)
- [Efficient Unlearning with Privacy Guarantees](https://mlsanity.com/paper/2507.04771/) (2025)
- [Unlearning at Scale: State-Exact Trace-Preserving Deletion in Billion-Parameter Language Models](https://mlsanity.com/paper/2508.12220/) (2025)
- [Fully Decentralized Certified Unlearning](https://mlsanity.com/paper/2512.08443/) (2025)
- [Statistical Unlearning of Distributions: A Hypothesis Testing Approach](https://mlsanity.com/paper/2605.16645/) (2026)
- [Auditing of Unlearning Algorithms](https://mlsanity.com/paper/2607.05898/) (2026)
- [Large Language Models in Cybersecurity: Applications, Vulnerabilities, and Defense Techniques](https://mlsanity.com/paper/2507.13629/) (2025)
- [Conformal Unlearning: A New Paradigm for Unlearning in Conformal Predictors](https://mlsanity.com/paper/2508.03245/) (2025)
- [Descend or Rewind? Stochastic Gradient Descent Unlearning](https://mlsanity.com/paper/2511.15983/) (2025)
- [Sequential Subspace Noise Injection Prevents Accuracy Collapse in Certified Unlearning](https://mlsanity.com/paper/2601.05134/) (2026)
- [A Robust Certified Machine Unlearning Method Under Distribution Shift](https://mlsanity.com/paper/2601.06967/) (2026)
- [Variance-Reduced (varepsilon,δ)-Unlearning using Forget Set Gradients](https://mlsanity.com/paper/2602.14938/) (2026)
- [Certified Per-Instance Unlearning Using Individual Sensitivity Bounds](https://mlsanity.com/paper/2602.15602/) (2026)
- [Unlearning with Asymmetric Sources: Improved Unlearning-Utility Trade-off with Public Data](https://mlsanity.com/paper/2605.11170/) (2026)
- [InstantForget: Update-Free Backdoor Unlearning with Inference-Time Feature Reset](https://mlsanity.com/paper/2606.15730/) (2026)
- [Auditing Machine Unlearning: A Systematic Research on Whether Models Truly Forget](https://mlsanity.com/paper/2606.16110/) (2026)
- [The Forgetting-Retention Dilemma: Certified Unlearning Theory in Continual Learning](https://mlsanity.com/paper/2606.29832/) (2026)
- [On Optimization Complexity of Second-Order Certified Unlearning](https://mlsanity.com/paper/2607.20192/) (2026)
- [When Is Deletion Ordering Tractable? From Update Dynamics to Permutation Structure](https://mlsanity.com/paper/2610.01149/) (2026)
- [From Mathematical to Executable Certificates for Machine Unlearning](https://mlsanity.com/paper/2610.02268/) (2026)
- [Certified Mechanistic Edits: Behavioral Guarantees for Skill Removal and Preservation](https://mlsanity.com/paper/2610.03502/) (2026)
- [Minimax Gaussian Mechanisms for Continual Machine Unlearning](https://mlsanity.com/paper/2610.11628/) (2026)

## Abstract (arXiv)

We address the problem of machine unlearning, where the goal is to remove the influence of specific training data from a model upon request, motivated by privacy concerns and regulatory requirements such as the "right to be forgotten." Unfortunately, existing methods rely on restrictive assumptions or lack formal guarantees. To this end, we propose a novel method for certified machine unlearning, leveraging the connection between unlearning and privacy amplification by stochastic post-processing. Our method uses noisy fine-tuning on the retain data, i.e., data that does not need to be removed, to ensure provable unlearning guarantees. This approach requires no assumptions about the underlying loss function, making it broadly applicable across diverse settings. We analyze the theoretical trade-offs in efficiency and accuracy and demonstrate empirically that our method not only achieves formal unlearning guarantees but also performs effectively in practice, outperforming existing baselines. Our code is available at https://github.com/stair-lab/certified-unlearning-neural-networks-icml-2025