# Langevin Unlearning: A New Perspective of Noisy Gradient Descent for Machine Unlearning

arXiv: https://arxiv.org/abs/2401.10371 · HTML: https://mlsanity.com/paper/2401.10371/

## Key facts

- Cited by 49 papers in the map (data through 14 Oct 2026).
- Placed in Machine unlearning, one of 12 directions in LLM safety and security.
- Builds on 33 of its 54 references that are in the map.
- First posted on arXiv 18 Jan 2024.

## Placed in

- LLM safety and security / [Machine unlearning](https://mlsanity.com/safety/machine-unlearning/)

## Builds on (most cited first, 30 of 33)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Deep Learning with Differential Privacy](https://mlsanity.com/paper/1607.00133/) (2016)
- [Machine Unlearning](https://mlsanity.com/paper/1912.03817/) (2019)
- [The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks](https://mlsanity.com/paper/1802.08232/) (2018)
- [Renyi Differential Privacy](https://mlsanity.com/paper/1702.07476/) (2017)
- [Train faster, generalize better: Stability of stochastic gradient descent](https://mlsanity.com/paper/1509.01240/) (2015)
- [Certified Data Removal from Machine Learning Models](https://mlsanity.com/paper/1911.03030/) (2019)
- [Remember What You Want to Forget: Algorithms for Machine Unlearning](https://mlsanity.com/paper/2103.03279/) (2021)
- [Descent-to-Delete: Gradient-Based Methods for Machine Unlearning](https://mlsanity.com/paper/2007.02923/) (2020)
- [The Composition Theorem for Differential Privacy](https://mlsanity.com/paper/1311.0776/) (2013)
- [Rapid Convergence of the Unadjusted Langevin Algorithm: Isoperimetry Suffices](https://mlsanity.com/paper/1903.08568/) (2019)
- [Non-asymptotic convergence analysis for the Unadjusted Langevin Algorithm](https://mlsanity.com/paper/1507.05021/) (2015)
- [Adaptive Machine Unlearning](https://mlsanity.com/paper/2106.04378/) (2021)
- [Practical and Private (Deep) Learning without Sampling or Shuffling](https://mlsanity.com/paper/2103.00039/) (2021)
- [Machine Unlearning via Algorithmic Stability](https://mlsanity.com/paper/2102.13179/) (2021)
- [Variational Bayesian Unlearning](https://mlsanity.com/paper/2010.12883/) (2020)
- [Privacy Amplification by Iteration](https://mlsanity.com/paper/1808.06651/) (2018)
- [Differential Privacy Dynamics of Langevin Diffusion and Noisy Gradient Descent](https://mlsanity.com/paper/2102.05855/) (2021)
- [Sampling Can Be Faster Than Optimization](https://mlsanity.com/paper/1811.08413/) (2018)
- [Privacy of Noisy Stochastic Gradient Descent: More Iterations without More Privacy Loss](https://mlsanity.com/paper/2205.13710/) (2022)
- [Forget Unlearning: Towards True Data-Deletion in Machine Learning](https://mlsanity.com/paper/2210.08911/) (2022)
- [Differentially Private Learning Needs Hidden State (Or Much Faster Convergence)](https://mlsanity.com/paper/2203.05363/) (2022)
- [Convergence of Langevin Monte Carlo in Chi-Squared and Renyi Divergence](https://mlsanity.com/paper/2007.11612/) (2020)
- [Projected Stochastic Gradient Langevin Algorithms for Constrained Sampling and Non-Convex Learning](https://mlsanity.com/paper/2012.12137/) (2020)
- [Towards a Complete Analysis of Langevin Monte Carlo: Beyond Poincar'e Inequality](https://mlsanity.com/paper/2303.03589/) (2023)
- [Faster Differentially Private Samplers via R'enyi Divergence Analysis of Discretized Langevin MCMC](https://mlsanity.com/paper/2010.14658/) (2020)
- [Markov Chain Monte Carlo-Based Machine Unlearning: Unlearning What Needs to be Forgotten](https://mlsanity.com/paper/2202.13585/) (2022)
- [Resolving the Mixing Time of the Langevin Algorithm to its Stationary Distribution for Log-Concave Sampling](https://mlsanity.com/paper/2210.08448/) (2022)
- [Differential Privacy Guarantees for Stochastic Gradient Langevin Dynamics](https://mlsanity.com/paper/2201.11980/) (2022)

## Built on by (most cited first, 30 of 49)

- [Machine Unlearning Fails to Remove Data Poisoning Attacks](https://mlsanity.com/paper/2406.17216/) (2024)
- [Certified Unlearning for Neural Networks](https://mlsanity.com/paper/2506.06985/) (2025)
- [Boosting Alignment for Post-Unlearning Text-to-Image Generative Models](https://mlsanity.com/paper/2412.07808/) (2024)
- [Certified Machine Unlearning via Noisy Stochastic Gradient Descent](https://mlsanity.com/paper/2403.17105/) (2024)
- [Do LLMs Really Forget? Evaluating Unlearning with Knowledge Correlation and Confidence Awareness](https://mlsanity.com/paper/2506.05735/) (2025)
- [Ascent Fails to Forget](https://mlsanity.com/paper/2509.26427/) (2025)
- [Underestimated Privacy Risks for Minority Populations in Large Language Model Unlearning](https://mlsanity.com/paper/2412.08559/) (2024)
- [A Certified Unlearning Approach without Access to Source Data](https://mlsanity.com/paper/2506.06486/) (2025)
- [Convergent Privacy Loss of Noisy-SGD without Convexity and Smoothness](https://mlsanity.com/paper/2410.01068/) (2024)
- [Towards Reliable Forgetting: A Survey on Machine Unlearning Verification](https://mlsanity.com/paper/2506.15115/) (2025)
- [The Right to be Forgotten in Pruning: Unveil Machine Unlearning on Sparse Models](https://mlsanity.com/paper/2507.18725/) (2025)
- [MUBox: A Critical Evaluation Framework of Deep Machine Unlearning](https://mlsanity.com/paper/2505.08576/) (2025)
- [Understanding Fine-tuning in Approximate Unlearning: A Theoretical Perspective](https://mlsanity.com/paper/2410.03833/) (2024)
- [Distributional Machine Unlearning via Selective Data Removal](https://mlsanity.com/paper/2507.15112/) (2025)
- [Provable unlearning in topic modeling and downstream tasks](https://mlsanity.com/paper/2411.12600/) (2024)
- [Leveraging Per-Instance Privacy for Machine Unlearning](https://mlsanity.com/paper/2505.18786/) (2025)
- [System-Aware Unlearning Algorithms: Use Lesser, Forget Faster](https://mlsanity.com/paper/2506.06073/) (2025)
- [The Unseen Threat: Residual Knowledge in Machine Unlearning under Perturbed Samples](https://mlsanity.com/paper/2601.22359/) (2026)
- [Privacy of SGD under Gaussian or Heavy-Tailed Noise: Guarantees without Gradient Clipping](https://mlsanity.com/paper/2403.02051/) (2024)
- [One-Point Contraction: Erasing Representational Separability toward Irreversible Deep Forgetting](https://mlsanity.com/paper/2507.07754/) (2025)
- [Quantum-Inspired Audio Unlearning: Towards Privacy-Preserving Voice Biometrics](https://mlsanity.com/paper/2507.22208/) (2025)
- [LLM Unlearning using Gradient Ratio-Based Influence Estimation and Noise Injection](https://mlsanity.com/paper/2508.06467/) (2025)
- [Representation Unlearning: Forgetting through Information Compression](https://mlsanity.com/paper/2601.21564/) (2026)
- [Efficient Unlearning with Privacy Guarantees](https://mlsanity.com/paper/2507.04771/) (2025)
- [Backdoor Unlearning by Linear Task Decomposition](https://mlsanity.com/paper/2510.14845/) (2025)
- [WARP: Weight Teleportation for Attack-Resilient Unlearning Protocols](https://mlsanity.com/paper/2512.00272/) (2025)
- [Efficient Public Verification of Private ML via Regularization](https://mlsanity.com/paper/2512.04008/) (2025)
- [Fully Decentralized Certified Unlearning](https://mlsanity.com/paper/2512.08443/) (2025)
- [Less Noise, Same Certificate: Retain Sensitivity for Unlearning](https://mlsanity.com/paper/2603.03172/) (2026)
- [Statistical Unlearning of Distributions: A Hypothesis Testing Approach](https://mlsanity.com/paper/2605.16645/) (2026)

## Abstract (arXiv)

Machine unlearning has raised significant interest with the adoption of laws ensuring the ``right to be forgotten''. Researchers have provided a probabilistic notion of approximate unlearning under a similar definition of Differential Privacy (DP), where privacy is defined as statistical indistinguishability to retraining from scratch. We propose Langevin unlearning, an unlearning framework based on noisy gradient descent with privacy guarantees for approximate unlearning problems. Langevin unlearning unifies the DP learning process and the privacy-certified unlearning process with many algorithmic benefits. These include approximate certified unlearning for non-convex problems, complexity saving compared to retraining, sequential and batch unlearning for multiple unlearning requests.