mlsanityData to 7 Oct 2026

Langevin Unlearning: A New Perspective of Noisy Gradient Descent for Machine Unlearning

arXiv 2401.10371 ↗First posted 18 Jan 2024cs.LGcited by 48

On the map: LLM safety and security → Machine unlearning

Abstract (arXiv)

Machine unlearning has raised significant interest with the adoption of laws ensuring the ``right to be forgotten''. Researchers have provided a probabilistic notion of approximate unlearning under a similar definition of Differential Privacy (DP), where privacy is defined as statistical indistinguishability to retraining from scratch. We propose Langevin unlearning, an unlearning framework based on noisy gradient descent with privacy guarantees for approximate unlearning problems. Langevin unlearning unifies the DP learning process and the privacy-certified unlearning process with many algorithmic benefits. These include approximate certified unlearning for non-convex problems, complexity saving compared to retraining, sequential and batch unlearning for multiple unlearning requests.

Key facts. Cited by 48 papers in the map (data through 7 Oct 2026). Placed in Machine unlearning, one of 12 directions in LLM safety and security. Builds on 33 of its 54 references that are in the map. First posted on arXiv 18 Jan 2024.

Builds on

33 of 54 references.

Closest ancestors · 7

Same direction, newest first

  1. 2023From Adaptive Query Release to Machine Unlearning20 Jul
  2. 2022Forget Unlearning: Towards True Data-Deletion in Machine Learning17 Oct
  3. 2022Challenges and Pitfalls of Bayesian Unlearning7 Jul
  4. 2022Markov Chain Monte Carlo-Based Machine Unlearning: Unlearning What Needs to be Forgotten28 Feb
  5. 2021Adaptive Machine Unlearning8 Jun
  6. 2021Remember What You Want to Forget: Algorithms for Machine Unlearning4 Mar
  7. 2021Machine Unlearning via Algorithmic Stability25 Feb

Other references · 26

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2016Deep Learning with Differential Privacycited by 2,682
  4. 2019Machine Unlearningcited by 1,020
  5. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 847
  6. 2017Renyi Differential Privacycited by 643
  7. 2015Train faster, generalize better: Stability of stochastic gradient descentcited by 547
  8. 2019Certified Data Removal from Machine Learning Modelscited by 512
  9. 2020Descent-to-Delete: Gradient-Based Methods for Machine Unlearningcited by 303
  10. 2013The Composition Theorem for Differential Privacycited by 290
  11. 2019Rapid Convergence of the Unadjusted Langevin Algorithm: Isoperimetry Sufficescited by 224
  12. 2015Non-asymptotic convergence analysis for the Unadjusted Langevin Algorithmcited by 195
14 more
  1. 2021Practical and Private (Deep) Learning without Sampling or Shufflingcited by 169
  2. 2020Variational Bayesian Unlearningcited by 114
  3. 2018Privacy Amplification by Iterationcited by 97
  4. 2021Differential Privacy Dynamics of Langevin Diffusion and Noisy Gradient Descentcited by 71
  5. 2018Sampling Can Be Faster Than Optimizationcited by 68
  6. 2022Privacy of Noisy Stochastic Gradient Descent: More Iterations without More Privacy Losscited by 60
  7. 2022Differentially Private Learning Needs Hidden State (Or Much Faster Convergence)cited by 45
  8. 2020Convergence of Langevin Monte Carlo in Chi-Squared and Renyi Divergencecited by 42
  9. 2020Projected Stochastic Gradient Langevin Algorithms for Constrained Sampling and Non-Convex Learningcited by 33
  10. 2023Towards a Complete Analysis of Langevin Monte Carlo: Beyond Poincar'e Inequalitycited by 31
  11. 2020Faster Differentially Private Samplers via R'enyi Divergence Analysis of Discretized Langevin MCMCcited by 27
  12. 2022Resolving the Mixing Time of the Langevin Algorithm to its Stationary Distribution for Log-Concave Samplingcited by 21
  13. 2022Differential Privacy Guarantees for Stochastic Gradient Langevin Dynamicscited by 18
  14. 2021Bayesian Inference Forgettingcited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 8; 2025: 27; 2026: 13.

48 papers.

Sort:

By area:

  1. 2024Machine Unlearning Fails to Remove Data Poisoning Attackscited by 35
  2. 2025Certified Unlearning for Neural Networkscited by 25
  3. 2024Boosting Alignment for Post-Unlearning Text-to-Image Generative Modelscited by 15
  4. 2024Certified Machine Unlearning via Noisy Stochastic Gradient Descentcited by 13
  5. 2025Do LLMs Really Forget? Evaluating Unlearning with Knowledge Correlation and Confidence Awarenesscited by 13
  6. 2025Ascent Fails to Forgetcited by 12
  7. 2024Underestimated Privacy Risks for Minority Populations in Large Language Model Unlearningcited by 9
  8. 2025A Certified Unlearning Approach without Access to Source Datacited by 9
  9. 2025Towards Reliable Forgetting: A Survey on Machine Unlearning Verificationcited by 8
  10. 2024Convergent Privacy Loss of Noisy-SGD without Convexity and Smoothnesscited by 6
  11. 2025MUBox: A Critical Evaluation Framework of Deep Machine Unlearningcited by 6
  12. 2024Understanding Fine-tuning in Approximate Unlearning: A Theoretical Perspectivecited by 5
36 more
  1. 2025Distributional Machine Unlearning via Selective Data Removalcited by 5
  2. 2025The Right to be Forgotten in Pruning: Unveil Machine Unlearning on Sparse Modelscited by 5
  3. 2024Provable unlearning in topic modeling and downstream taskscited by 4
  4. 2025Leveraging Per-Instance Privacy for Machine Unlearningcited by 4
  5. 2026The Unseen Threat: Residual Knowledge in Machine Unlearning under Perturbed Samplescited by 4
  6. 2024Privacy of SGD under Gaussian or Heavy-Tailed Noise: Guarantees without Gradient Clippingcited by 3
  7. 2025System-Aware Unlearning Algorithms: Use Lesser, Forget Fastercited by 3
  8. 2025Quantum-Inspired Audio Unlearning: Towards Privacy-Preserving Voice Biometricscited by 2
  9. 2025Efficient Unlearning with Privacy Guaranteescited by 1
  10. 2025One-Point Contraction: Erasing Representational Separability toward Irreversible Deep Forgettingcited by 1
  11. 2025LLM Unlearning using Gradient Ratio-Based Influence Estimation and Noise Injectioncited by 1
  12. 2025Backdoor Unlearning by Linear Task Decompositioncited by 1
  13. 2025WARP: Weight Teleportation for Attack-Resilient Unlearning Protocolscited by 1
  14. 2025Efficient Public Verification of Private ML via Regularizationcited by 1
  15. 2025Fully Decentralized Certified Unlearningcited by 1
  16. 2026Representation Unlearning: Forgetting through Information Compressioncited by 1
  17. 2026Less Noise, Same Certificate: Retain Sensitivity for Unlearningcited by 1
  18. 2026Auditing of Unlearning Algorithmscited by 1
  19. 2025Task-Agnostic Federation over Decentralized Data: Research Landscape and Visionscited by 0
  20. 2025Targeted Unlearning Using Perturbed Sign Gradient Methods With Applications On Medical Imagescited by 0
  21. 2025Privacy Amplification in Differentially Private Zeroth-Order Optimization with Hidden Statescited by 0
  22. 2025Machine Unlearning for Streaming Forgettingcited by 0
  23. 2025Graph Unlearning via Embedding Reconstruction -- A Range-Null Space Decomposition Approachcited by 0
  24. 2025The Measure of Deception: An Analysis of Data Forging in Machine Unlearningcited by 0
  25. 2025Factor Decorrelation Enhanced Data Removal from Deep Predictive Modelscited by 0
  26. 2025R'enyi Differential Privacy for Heavy-Tailed SDEs via Fractional Poincar'e Inequalitiescited by 0
  27. 2025Descend or Rewind? Stochastic Gradient Descent Unlearningcited by 0
  28. 2026Sequential Subspace Noise Injection Prevents Accuracy Collapse in Certified Unlearningcited by 0
  29. 2026Why Some Models Resist Unlearning: A Linear Stability Perspectivecited by 0
  30. 2026Variance-Reduced (varepsilon,δ)-Unlearning using Forget Set Gradientscited by 0
  31. 2026Certified Per-Instance Unlearning Using Individual Sensitivity Boundscited by 0
  32. 2026Behavioral Guarantees for Proxy-Based Unlearningcited by 0
  33. 2026Unlearning with Asymmetric Sources: Improved Unlearning-Utility Trade-off with Public Datacited by 0
  34. 2026SoK: Unlearnability and Unlearning for Model Dememorizationcited by 0
  35. 2026Statistical Unlearning of Distributions: A Hypothesis Testing Approachcited by 0
  36. 2026The Forgetting-Retention Dilemma: Certified Unlearning Theory in Continual Learningcited by 0