mlsanityData to 14 Oct 2026

A Survey on Unlearning in Large Language Models

arXiv 2510.25117 ↗First posted 29 Oct 2025cs.CLcited by 4

On the map: LLM safety and security → Machine unlearning

Abstract (arXiv)

Large Language Models (LLMs) demonstrate remarkable capabilities, but their training on massive corpora poses significant risks from memorized sensitive information. To mitigate these issues and align with legal standards, unlearning has emerged as a critical technique to selectively erase specific knowledge from LLMs without compromising their overall performance. This survey provides a systematic review of over 180 papers on LLM unlearning published since 2021. First, it introduces a novel taxonomy that categorizes unlearning methods based on the phase in the LLM pipeline of the intervention. This framework further distinguishes between parameter modification and parameter selection strategies, thus enabling deeper insights and more informed comparative analysis. Second, it offers a multidimensional analysis of evaluation paradigms. For datasets, we compare 18 existing benchmarks from the perspectives of task format, content, and experimental paradigms to offer actionable guidance. For metrics, we move beyond mere enumeration by dividing knowledge memorization metrics into 10 categories to analyze their advantages and applicability, while also reviewing metrics for model utility, robustness, and efficiency. By discussing current challenges and future directions, this survey aims to advance the field of LLM unlearning and the development of secure AI systems.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in Machine unlearning, one of 12 directions in LLM safety and security. Builds on 154 of its 199 references that are in the map. First posted on arXiv 29 Oct 2025.

Builds on

154 of 199 references.

Closest ancestors · 90

Same direction, newest first

  1. 2025Unlearning That Lasts: Utility-Preserving, Robust, and Almost Irreversible Forgetting in LLMs2 Sep
  2. 2025RULE: Reinforcement UnLEarning Achieves Forget-Retain Pareto Optimality8 Jun
  3. 2025Rethinking LLM Unlearning Objectives: A Gradient Perspective and Go Beyond26 Feb
  4. 2025FaithUn: Toward Faithful Forgetting in Language Models by Investigating the Interconnectedness of Knowledge26 Feb
  5. 2025Holistic Audit Dataset Generation for LLM Unlearning via Knowledge Graph Traversal and Redundancy Removal26 Feb
  6. 2025A General Framework to Enhance Fine-tuning-based LLM Unlearning25 Feb
  7. 2025CoME: An Unlearning-based Approach to Conflict-free Model Editing20 Feb
  8. 2025LUME: LLM Unlearning with Multitask Evaluations20 Feb
  9. 2025UPCORE: Utility-Preserving Coreset Selection for Balanced Unlearning20 Feb
  10. 2025Obliviate: Efficient Unmemorization for Protecting Intellectual Property in Large Language Models20 Feb
  11. 2025Beyond Single-Value Metrics: Evaluating and Enhancing LLM Unlearning with Cognitive Diagnosis19 Feb
  12. 2025Which Retain Set Matters for LLM Unlearning? A Case Study on Entity Unlearning17 Feb
38 more
  1. 2025ReLearn: Unlearning via Learning for Large Language Models16 Feb
  2. 2025LLM Unlearning via Neural Activation Redirection11 Feb
  3. 2025Mitigating Sensitive Information Leakage in LLMs4Code through Machine Unlearning9 Feb
  4. 2025Towards LLM Unlearning Resilient to Relearning Attacks: A Sharpness-Aware Minimization Perspective and Beyond7 Feb
  5. 2025Tool Unlearning for Tool-Augmented LLMs3 Feb
  6. 2025Agents Are All You Need for LLM Unlearning1 Feb
  7. 2025Resolving Editing-Unlearning Conflicts: A Knowledge Codebook Framework for Large Language Model Updating31 Jan
  8. 2025Improving LLM Unlearning Robustness via Random Perturbations31 Jan
  9. 2025Open Problems in Machine Unlearning for AI Safety9 Jan
  10. 2024Multi-Objective Large Language Model Unlearning29 Dec
  11. 2024Underestimated Privacy Risks for Minority Populations in Large Language Model Unlearning11 Dec
  12. 2024SEUF: Is Unlearning One Expert Enough for Mixture-of-Experts LLMs?27 Nov
  13. 2024Towards Robust Evaluation of Unlearning in LLMs via Data Transformations23 Nov
  14. 2024Does Unlearning Truly Unlearn? A Black Box Evaluation of LLM Unlearning Methods18 Nov
  15. 2024Extracting Unlearned Information from LLMs with Activation Steering4 Nov
  16. 2024Unlearning as multi-task optimization: A normalized gradient difference approach with an adaptive learning rate29 Oct
  17. 2024Learning and Unlearning of Fabricated Knowledge in Language Models29 Oct
  18. 2024Applying sparse autoencoders to unlearn knowledge in language models25 Oct
  19. 2024UnStar: Unlearning with Self-Taught Anti-Sample Reasoning for LLMs22 Oct
  20. 2024Catastrophic Failure of LLM Unlearning via Quantization21 Oct
  21. 2024When Machine Unlearning Meets Retrieval-Augmented Generation (RAG): Keep Secret or Forget Knowledge?20 Oct
  22. 2024Breaking Chains: Unraveling the Links in Multi-Hop Knowledge Unlearning17 Oct
  23. 2024Mechanistic Unlearning: Robust Knowledge Unlearning and Editing via Mechanistic Localization16 Oct
  24. 2024LLM Unlearning via Loss Adjustment with Only Forget Data14 Oct
  25. 2024Do Unlearning Methods Remove Information from Language Model Weights?11 Oct
  26. 2024CodeUnlearn: Amortized Zero-Shot Machine Unlearning in Language Models Using Discrete Concept8 Oct
  27. 2024Answer When Needed, Forget When Not: Language Models Pretend to Forget via In-Context Knowledge Unlearning1 Oct
  28. 2024An Adversarial Perspective on Machine Unlearning for AI Safety26 Sep
  29. 2024MEOW: MEMOry Supervised LLM Unlearning Via Inverted Facts18 Sep
  30. 2024WPN: An Unlearning Method Based on N-pair Contrastive Learning in Language Models18 Aug
  31. 2024On Effects of Steering Latent Representation for Large Language Model Unlearning12 Aug
  32. 2024On the Limitations and Prospects of Machine Unlearning for Generative AI1 Aug
  33. 2024Machine Unlearning in Generative AI: A Survey30 Jul
  34. 2024Learn while Unlearn: An Iterative Unlearning Framework for Generative Language Models25 Jul
  35. 2024Revisiting Who's Harry Potter: Towards Targeted Unlearning from a Causal Intervention Perspective24 Jul
  36. 2024On Large Language Model Continual Unlearning14 Jul
  37. 2024Learning to Refuse: Towards Mitigating Privacy Risks in LLMs14 Jul
  38. 2024MUSE: Machine Unlearning Six-Way Evaluation for Language Models8 Jul

50 newest of 90 All 90 (JSON)

Other references · 64

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  4. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  5. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  6. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  7. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  8. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  9. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
  10. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
  11. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  12. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
38 more
  1. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
  2. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,883
  3. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  4. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,411
  5. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,281
  6. 2020Extracting Training Data from Large Language Modelscited by 2,189
  7. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,164
  8. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,799
  9. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,786
  10. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,420
  11. 2016Understanding intermediate layers using linear classifier probescited by 1,353
  12. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,336
  13. 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,203
  14. 2024The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scalecited by 1,160
  15. 2019Machine Unlearningcited by 1,061
  16. 2016The LAMBADA dataset: Word prediction requiring a broad discourse contextcited by 992
  17. 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 876
  18. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 864
  19. 2023BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Datasetcited by 860
  20. 2025SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-trainingcited by 672
  21. 2019MathQA: Towards Interpretable Math Word Problem Solving with Operation-Based Formalismscited by 662
  22. 2023Dissecting Recall of Factual Associations in Auto-Regressive Language Modelscited by 528
  23. 2024The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearningcited by 509
  24. 2019Neural Text Generation with Unlikelihood Trainingcited by 503
  25. 2021MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontierscited by 470
  26. 2021DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Expertscited by 441
  27. 2023Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrationscited by 431
  28. 2017Zero-Shot Relation Extraction via Reading Comprehensioncited by 426
  29. 2023Editing Large Language Models: Problems, Methods, and Opportunitiescited by 372
  30. 2023LEACE: Perfect linear concept erasure in closed formcited by 267
  31. 2024PKU-SafeRLHF: Towards Multi-Level Safety Alignment for LLMs with Human Preferencecited by 261
  32. 2022Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language Modelscited by 220
  33. 2023Scalable and Transferable Black-Box Jailbreaks for Language Models via Persona Modulationcited by 214
  34. 2023Localizing Model Behavior with Path Patchingcited by 195
  35. 2023LoRA Fine-tuning Efficiently Undoes Safety Training in Llama 2-Chat 70Bcited by 195
  36. 2023Removing RLHF Protections in GPT-4 via Fine-Tuningcited by 171
  37. 2018Canonical Tensor Decomposition for Knowledge Base Completioncited by 150
  38. 2015Adding Gradient Noise Improves Learning for Very Deep Networkscited by 145

50 most cited of 64 All 64 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026LLMs Can Unlearn Refusal with Only 1,000 Benign Samplescited by 3
  2. 2026MPU: Towards Secure and Privacy-Preserving Knowledge Unlearning for Large Language Modelscited by 0
  3. 2026ConceptGuard: Benchmarking Context-Sensitive Unlearning in Large Language Modelscited by 0
  4. 2026RA2: Retain-Anchored Attraction Preserves Forget-Adjacent Utility in LLM Unlearningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Unlearning in LLMs: Methods, Evaluation, and Open Challenges
  2. 2025A Comprehensive Survey of Machine Unlearning Techniques for Large Language Models
  3. 2025LLM Unlearning Under the Microscope: A Full-Stack View on Methods and Metrics
  4. 2024To Forget or Not? Towards Practical Knowledge Unlearning for Large Language Models
  5. 2024A Closer Look at Machine Unlearning for Large Language Models
  6. 2025Not Every Token Needs Forgetting: Selective Unlearning to Limit Change in Utility in Large Language Model Unlearning
  7. 2026The Unlearning Mirage: A Dynamic Framework for Evaluating LLM Unlearning
  8. 2023Knowledge Unlearning for LLMs: Tasks, Methods, and Challenges