mlsanityData to 14 Oct 2026

SoK: Machine Unlearning for Large Language Models

arXiv 2506.09227 ↗First posted 10 Jun 2025cs.LG cs.CRcited by 11

On the map: LLM safety and security → Machine unlearning

Abstract (arXiv)

Large language model (LLM) unlearning has become a critical topic in machine learning, aiming to eliminate the influence of specific training data or knowledge without retraining the model from scratch. A variety of techniques have been proposed, including Gradient Ascent, model editing, and re-steering hidden representations. While existing surveys often organize these methods by their technical characteristics, such classifications tend to overlook a more fundamental dimension: the underlying intention of unlearning--whether it seeks to truly remove internal knowledge or merely suppress its behavioral effects. In this SoK paper, we propose a new taxonomy based on this intention-oriented perspective. Building on this taxonomy, we make three key contributions. First, we revisit recent findings suggesting that many removal methods may functionally behave like suppression, and explore whether true removal is necessary or achievable. Second, we survey existing evaluation strategies, identify limitations in current metrics and benchmarks, and suggest directions for developing more reliable and intention-aligned evaluations. Third, we highlight practical challenges--such as scalability and support for sequential unlearning--that currently hinder the broader deployment of unlearning methods. In summary, this work offers a comprehensive framework for understanding and advancing unlearning in generative AI, aiming to support future research and guide policy decisions around data removal and privacy.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Placed in Machine unlearning, one of 12 directions in LLM safety and security. Builds on 104 of its 118 references that are in the map. First posted on arXiv 10 Jun 2025.

Builds on

104 of 118 references.

Closest ancestors · 78

Same direction, newest first

  1. 2025Keeping an Eye on LLM Unlearning: The Hidden Risk and Remedy31 May
  2. 2025Harry Potter is Still Here! Probing Knowledge Leakage in Targeted Unlearned Large Language Models via Automated Adversarial Prompting22 May
  3. 2025Unlearning Isn't Deletion: Investigating Reversibility of Machine Unlearning in LLMs22 May
  4. 2025Does Localization Inform Unlearning? A Rigorous Examination of Local Parameter Attribution for Knowledge Unlearning in Language Models22 May
  5. 2025Losing is for Cherishing: Data Valuation Based on Machine Unlearning and Shapley Value22 May
  6. 2025R-TOFU: Unlearning in Large Reasoning Models21 May
  7. 2025DUSK: Do Not Unlearn Shared Knowledge21 May
  8. 2025A Unified Gradient-based Framework for Task-agnostic Continual Learning-Unlearning21 May
  9. 2025GUARD: Generation-time LLM Unlearning via Adaptive Restriction and Detection19 May
  10. 2025DualOptim: Enhancing Efficacy and Stability in Machine Unlearning with Dual Optimizers22 Apr
  11. 2025GRAIL: Gradient-Based Adaptive Unlearning for Privacy and Copyright in LLMs17 Apr
  12. 2025When is Task Vector Provably Effective for Model Editing? A Generalization Analysis of Nonlinear Transformers15 Apr
38 more
  1. 2025LLM Unlearning Reveals a Stronger-Than-Expected Coreset Effect in Current Benchmarks14 Apr
  2. 2025SAEs Can Improve Unlearning: Dynamic Sparse Autoencoder Guardrails for Precision Unlearning in LLMs11 Apr
  3. 2025Not All Data Are Unlearned Equally7 Apr
  4. 2025Exact Unlearning of Finetuning Data via Model Merging at Scale6 Apr
  5. 2025SUV: Scalable Large Language Model Copyright Compliance with Regularized Selective Unlearning29 Mar
  6. 2025Effective Skill Unlearning through Intervention and Abstention27 Mar
  7. 2025SAUCE: Selective Concept Unlearning in Vision-Language Models with Sparse Autoencoders16 Mar
  8. 2025Don't Forget It! Conditional Sparse Autoencoder Clamping Works for Unlearning14 Mar
  9. 2025Erasing Without Remembering: Implicit Knowledge Forgetting in Large Language Models27 Feb
  10. 2025Rethinking LLM Unlearning Objectives: A Gradient Perspective and Go Beyond26 Feb
  11. 2025FaithUn: Toward Faithful Forgetting in Language Models by Investigating the Interconnectedness of Knowledge26 Feb
  12. 2025A General Framework to Enhance Fine-tuning-based LLM Unlearning25 Feb
  13. 2025Modality-Aware Neuron Pruning for Unlearning in Multimodal Large Language Models21 Feb
  14. 2025Soft Token Attacks Cannot Reliably Audit Unlearning in Large Language Models20 Feb
  15. 2025LUME: LLM Unlearning with Multitask Evaluations20 Feb
  16. 2025UPCORE: Utility-Preserving Coreset Selection for Balanced Unlearning20 Feb
  17. 2025Obliviate: Efficient Unmemorization for Protecting Intellectual Property in Large Language Models20 Feb
  18. 2025Which Retain Set Matters for LLM Unlearning? A Case Study on Entity Unlearning17 Feb
  19. 2025ReLearn: Unlearning via Learning for Large Language Models16 Feb
  20. 2025LLM Unlearning via Neural Activation Redirection11 Feb
  21. 2025FALCON: Fine-grained Activation Manipulation by Contrastive Orthogonal Unalignment for Large Language Model3 Feb
  22. 2025Agents Are All You Need for LLM Unlearning1 Feb
  23. 2024Unlearning as multi-task optimization: A normalized gradient difference approach with an adaptive learning rate29 Oct
  24. 2024Applying sparse autoencoders to unlearn knowledge in language models25 Oct
  25. 2024Catastrophic Failure of LLM Unlearning via Quantization21 Oct
  26. 2024When Machine Unlearning Meets Retrieval-Augmented Generation (RAG): Keep Secret or Forget Knowledge?20 Oct
  27. 2024Mechanistic Unlearning: Robust Knowledge Unlearning and Editing via Mechanistic Localization16 Oct
  28. 2024Do Unlearning Methods Remove Information from Language Model Weights?11 Oct
  29. 2024A Closer Look at Machine Unlearning for Large Language Models10 Oct
  30. 2024Simplicity Prevails: Rethinking Negative Preference Optimization for LLM Unlearning9 Oct
  31. 2024Dissecting Fine-Tuning Unlearning in Large Language Models9 Oct
  32. 2024NegMerge: Sign-Consensual Weight Merging for Machine Unlearning8 Oct
  33. 2024A Probabilistic Perspective on Unlearning and Alignment for Large Language Models4 Oct
  34. 2024Position: LLM Unlearning Benchmarks are Weak Measures of Progress3 Oct
  35. 2024Answer When Needed, Forget When Not: Language Models Pretend to Forget via In-Context Knowledge Unlearning1 Oct
  36. 2024LLM Surgery: Efficient Knowledge Unlearning and Editing in Large Language Models19 Sep
  37. 2024Towards Robust Knowledge Unlearning: An Adversarial Framework for Assessing and Improving Unlearning Robustness in Large Language Models20 Aug
  38. 2024Towards Robust and Parameter-Efficient Knowledge Unlearning for LLMs13 Aug

50 newest of 78 All 78 (JSON)

Other references · 26

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  3. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  4. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  5. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  6. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  7. 2020Extracting Training Data from Large Language Modelscited by 2,189
  8. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,420
  9. 2022Editing Models with Task Arithmeticcited by 1,120
  10. 2019Machine Unlearningcited by 1,061
  11. 2023An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuningcited by 639
  12. 2024The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearningcited by 509
14 more
  1. 2023Detecting Pretraining Data from Large Language Modelscited by 387
  2. 2024Improving Alignment and Robustness with Circuit Breakerscited by 322
  3. 2020Amnesiac Machine Learningcited by 319
  4. 2016Deep Reinforcement Learning for Dialogue Generationcited by 288
  5. 2023Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-trainingcited by 265
  6. 2023The Unlocking Spell on Base LLMs: Rethinking Alignment via In-Context Learningcited by 257
  7. 2024Assessing the Brittleness of Safety Alignment via Pruning and Low-Rank Modificationscited by 231
  8. 2023Regulating ChatGPT and other Large Generative AI Modelscited by 104
  9. 2019Neural Machine Translation: A Review and Surveycited by 100
  10. 2024Soft Prompt Threats: Attacking Safety Alignment and Unlearning in Open-Source LLMs through the Embedding Spacecited by 91
  11. 2025A Survey on Sparse Autoencoders: Interpreting the Internal Mechanisms of Large Language Modelscited by 76
  12. 2024Copyright Protection in Generative AI: A Technical Perspectivecited by 38
  13. 2023Knowledge Sanitization of Large Language Modelscited by 37
  14. 2025(De)-Indexing and the Right to be Forgottencited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 9.

11 papers.

  1. 2026REBEL: Hidden Knowledge Recovery via Evolutionary-Based Evaluation Loopcited by 4
  2. 2026KUDA: Knowledge Unlearning by Deviating Representation for Large Language Modelscited by 2
  3. 2026Beyond Forgetting: Representation Misdirection Elicits Controllable Side Behaviors and Capabilitiescited by 1
  4. 2026Selective Capability Unlearning in End-to-End Spoken Language Understandingcited by 1
  5. 2025SA-ADP: Sensitivity-Aware Adaptive Differential Privacy for Large Language Modelscited by 0
  6. 2025RapidUn: Influence-Driven Parameter Reweighting for Efficient Large Language Model Unlearningcited by 0
  7. 2026EvoMU: Evolutionary Machine Unlearningcited by 0
  8. 2026Relationship-Aware Safety Unlearning for Multimodal LLMscited by 0
  9. 2026Probe-Geometry Alignment: Erasing the Cross-Sequence Memorization Signature Below Chancecited by 0
  10. 2026LLM unbranding: Erasing Commercial Identity while Preserving Generic Utilitycited by 0
  11. 2026SIEVE: Selective attention-value Suppression for Vision-Language Models Unlearningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Unlearning in LLMs: Methods, Evaluation, and Open Challenges
  2. 2025A Comprehensive Survey of Machine Unlearning Techniques for Large Language Models
  3. 2024Multi-Objective Large Language Model Unlearning
  4. 2025A Survey on Unlearning in Large Language Models
  5. 2025LLM Unlearning Under the Microscope: A Full-Stack View on Methods and Metrics
  6. 2024Towards Effective Evaluations and Comparisons for LLM Unlearning Methods
  7. 2024Machine Unlearning in Large Language Models
  8. 2024The Frontier of Data Erasure: Machine Unlearning for Large Language Models