mlsanityData to 7 Oct 2026

Eight Methods to Evaluate Robust Unlearning in LLMs

arXiv 2402.16835 ↗First posted 26 Feb 2024cs.CLcited by 153

On the map: LLM safety and security → Machine unlearning

Abstract (arXiv)

Machine unlearning can be useful for removing harmful capabilities and memorized text from large language models (LLMs), but there are not yet standardized methods for rigorously evaluating it. In this paper, we first survey techniques and limitations of existing unlearning evaluations. Second, we apply a comprehensive set of tests for the robustness and competitiveness of unlearning in the "Who's Harry Potter" (WHP) model from Eldan and Russinovich (2023). While WHP's unlearning generalizes well when evaluated with the "Familiarity" metric from Eldan and Russinovich, we find i) higher-than-baseline amounts of knowledge can reliably be extracted, ii) WHP performs on par with the original model on Harry Potter Q&A tasks, iii) it represents latent knowledge comparably to the original model, and iv) there is collateral unlearning in related domains. Overall, our results highlight the importance of comprehensive unlearning evaluation that avoids ad-hoc metrics.

Key facts. Cited by 153 papers in the map (data through 7 Oct 2026). Placed in Machine unlearning, one of 12 directions in LLM safety and security. Builds on 59 of its 63 references that are in the map. First posted on arXiv 26 Feb 2024.

Builds on

59 of 63 references.

Closest ancestors · 12

Same direction, newest first

  1. 2024Towards Safer Large Language Models through Machine Unlearning15 Feb
  2. 2024TOFU: A Task of Fictitious Unlearning for LLMs11 Jan
  3. 2023Knowledge Unlearning for LLMs: Tasks, Methods, and Challenges27 Nov
  4. 2023Unlearn What You Want to Forget: Efficient Unlearning for LLMs31 Oct
  5. 2023In-Context Unlearning: Language Models as Few Shot Unlearners11 Oct
  6. 2023Who's Harry Potter? Approximate Unlearning in LLMs3 Oct
  7. 2023KGA: A General Machine Unlearning Framework Based on Knowledge Gap Alignment11 May
  8. 2023Exploring the Landscape of Machine Unlearning: A Comprehensive Survey and Taxonomy10 May
  9. 2022Privacy Adhering Machine Un-learning in NLP19 Dec
  10. 2022Knowledge Unlearning for Mitigating Privacy Risks in Language Models4 Oct
  11. 2022A Survey of Machine Unlearning6 Sep
  12. 2022Quark: Controllable Text Generation with Reinforced Unlearning26 May

Other references · 47

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  5. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
  6. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,239
  7. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,106
  8. 2022Editing Models with Task Arithmeticcited by 1,076
  9. 2019Machine Unlearningcited by 1,020
  10. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,009
  11. 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 831
  12. 2023Steering Language Models With Activation Engineeringcited by 824
35 more
  1. 2022Quantifying Memorization Across Neural Language Modelscited by 755
  2. 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 733
  3. 2019Certified Data Removal from Machine Learning Modelscited by 512
  4. 2023Jailbreaking ChatGPT via Prompt Engineering: An Empirical Studycited by 478
  5. 2024Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Trainingcited by 478
  6. 2023"Do Anything Now": Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Modelscited by 457
  7. 2023Detecting Pretraining Data from Large Language Modelscited by 375
  8. 2023Language Models Represent Space and Timecited by 306
  9. 2023Low-Resource Languages Jailbreak GPT-4cited by 290
  10. 2023Shadow Alignment: The Ease of Subverting Safely-Aligned Language Modelscited by 284
  11. 2024Assessing the Brittleness of Safety Alignment via Pruning and Low-Rank Modificationscited by 220
  12. 2023Scalable and Transferable Black-Box Jailbreaks for Language Models via Persona Modulationcited by 202
  13. 2023Survey of Vulnerabilities in Large Language Models Revealed by Adversarial Attackscited by 189
  14. 2023LoRA Fine-tuning Efficiently Undoes Safety Training in Llama 2-Chat 70Bcited by 183
  15. 2024A Mechanistic Understanding of Alignment Algorithms: A Case Study on DPO and Toxicitycited by 179
  16. 2023Removing RLHF Protections in GPT-4 via Fine-Tuningcited by 165
  17. 2023Can Sensitive Information Be Deleted From LLMs? Objectives for Defending Against Extraction Attackscited by 156
  18. 2023Composing Parameter-Efficient Modules with Arithmetic Operationscited by 155
  19. 2022Fine-tuned Language Models are Continual Learnerscited by 152
  20. 2024Fine-Tuning Enhances Existing Mechanisms: A Case Study on Entity Trackingcited by 140
  21. 2023Understanding Catastrophic Forgetting in Language Models via Implicit Inferencecited by 138
  22. 2023Mechanistically analyzing the effects of fine-tuning on procedurally defined taskscited by 110
  23. 2023Copyright Violations and Large Language Modelscited by 108
  24. 2024Soft Prompt Threats: Attacking Safety Alignment and Unlearning in Open-Source LLMs through the Embedding Spacecited by 88
  25. 2022Continual Pre-Training Mitigates Forgetting in Language and Visioncited by 82
  26. 2023DEPN: Detecting and Editing Privacy Neurons in Pretrained Language Modelscited by 75
  27. 2023Cognitive Dissonance: Why Do Language Model Outputs Disagree with Internal Representations of Truthfulness?cited by 62
  28. 2022Self-Destructing Models: Increasing the Costs of Harmful Dual Uses of Foundation Modelscited by 56
  29. 2022Linear Connectivity Reveals Generalization Strategiescited by 55
  30. 2022Mechanistic Mode Connectivitycited by 50
  31. 2023Knowledge Sanitization of Large Language Modelscited by 34
  32. 2024Large Language Models Relearn Removed Conceptscited by 26
  33. 2022Technical Report for ICCV 2021 Challenge SSLAD-Track3B: Transformers Are Better Continual Learnerscited by 20
  34. 2023Investigating Forgetting in Pre-Trained Representations Through Continual Learningcited by 19
  35. 2024Investigating Bias Representations in Llama 2 Chat via Activation Steeringcited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 46; 2025: 67; 2026: 40.

153 papers.

Sort:

By area:

  1. 2024The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearningcited by 488
  2. 2024Negative Preference Optimization: From Catastrophic Collapse to Effective Unlearningcited by 484
  3. 2024MUSE: Machine Unlearning Six-Way Evaluation for Language Modelscited by 251
  4. 2024Simplicity Prevails: Rethinking Negative Preference Optimization for LLM Unlearningcited by 160
  5. 2024Latent Adversarial Training Improves Robustness to Persistent Harmful Behaviors in LLMscited by 158
  6. 2024LLM Defenses Are Not Robust to Multi-Turn Human Jailbreaks Yetcited by 145
  7. 2024Tamper-Resistant Safeguards for Open-Weight LLMscited by 132
  8. 2024AI Sandbagging: Language Models can Strategically Underperform on Evaluationscited by 131
  9. 2024A Practical Review of Mechanistic Interpretability for Transformer-Based Language Modelscited by 113
  10. 2024An Adversarial Perspective on Machine Unlearning for AI Safetycited by 113
  11. 2024Large Language Model Unlearning via Embedding-Corrupted Promptscited by 112
  12. 2025The Attacker Moves Second: Stronger Adaptive Attacks Bypass Defenses Against Llm Jailbreaks and Prompt Injectionscited by 108
38 more
  1. 2024SOUL: Unlocking the Power of Second-Order Optimization for LLM Unlearningcited by 104
  2. 2024Rethinking LLM Memorization through the Lens of Adversarial Compressioncited by 91
  3. 2024Guardrail Baselines for Unlearning in LLMscited by 90
  4. 2024Unlearning or Obfuscating? Jogging the Memory of Unlearned LLMs via Benign Relearningcited by 81
  5. 2025OpenUnlearning: Accelerating LLM Unlearning via Unified Benchmarking of Methods and Metricscited by 80
  6. 2024RWKU: Benchmarking Real-World Knowledge Unlearning for Large Language Modelscited by 77
  7. 2024Catastrophic Failure of LLM Unlearning via Quantizationcited by 76
  8. 2025Towards LLM Unlearning Resilient to Relearning Attacks: A Sharpness-Aware Minimization Perspective and Beyondcited by 76
  9. 2024Do Unlearning Methods Remove Information from Language Model Weights?cited by 66
  10. 2024Position: LLM Unlearning Benchmarks are Weak Measures of Progresscited by 58
  11. 2024Demystifying Verbatim Memorization in Large Language Modelscited by 55
  12. 2024What Makes and Breaks Safety Fine-tuning? A Mechanistic Studycited by 54
  13. 2024Machine Unlearning in Generative AI: A Surveycited by 49
  14. 2024UnUnlearning: Unlearning is not sufficient for content regulation in advanced generative AIcited by 46
  15. 2024Stress-Testing Capability Elicitation With Password-Locked Modelscited by 42
  16. 2025Unlearning Isn't Deletion: Investigating Reversibility of Machine Unlearning in LLMscited by 41
  17. 2025Model Tampering Attacks Enable More Rigorous Evaluations of LLM Capabilitiescited by 36
  18. 2024Threats, Attacks, and Defenses in Machine Unlearning: A Surveycited by 34
  19. 2024Erasing Conceptual Knowledge from Language Modelscited by 34
  20. 2024Mechanistic Unlearning: Robust Knowledge Unlearning and Editing via Mechanistic Localizationcited by 34
  21. 2024Revisiting Who's Harry Potter: Towards Targeted Unlearning from a Causal Intervention Perspectivecited by 33
  22. 2025A Comprehensive Survey of Machine Unlearning Techniques for Large Language Modelscited by 30
  23. 2024Intrinsic Test of Unlearning Using Parametric Knowledge Tracescited by 28
  24. 2025Representation Bending for Large Language Model Safetycited by 28
  25. 2024Towards Effective Evaluations and Comparisons for LLM Unlearning Methodscited by 27
  26. 2024On Effects of Steering Latent Representation for Large Language Model Unlearningcited by 27
  27. 2024WAGLE: Strategic Weight Attribution for Effective and Modular Unlearning in Large Language Modelscited by 25
  28. 2025BLUR: A Bi-Level Optimization Approach for LLM Unlearningcited by 23
  29. 2025A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Modelscited by 22
  30. 2024A Probabilistic Perspective on Unlearning and Alignment for Large Language Modelscited by 21
  31. 2025Invariance Makes LLM Unlearning Resilient Even to Unanticipated Downstream Fine-Tuningcited by 20
  32. 2024Extracting Unlearned Information from LLMs with Activation Steeringcited by 19
  33. 2024Unbridled Icarus: A Survey of the Potential Perils of Image Inputs in Multimodal Large Language Model Securitycited by 18
  34. 2024Representation Shattering in Transformers: A Synthetic Study with Knowledge Editingcited by 18
  35. 2024Exploring Memorization and Copyright Violation in Frontier LLMs: A Study of the New York Times v. OpenAI 2023 Lawsuitcited by 17
  36. 2025Tool Unlearning for Tool-Augmented LLMscited by 17
  37. 2025GUARD: Generation-time LLM Unlearning via Adaptive Restriction and Detectioncited by 17
  38. 2025LLM Unlearning with LLM Beliefscited by 17

50 most cited of 153 All 153 (JSON)