mlsanityData to 7 Oct 2026

Unlearning or Obfuscating? Jogging the Memory of Unlearned LLMs via Benign Relearning

arXiv 2406.13356 ↗First posted 19 Jun 2024cs.LGcited by 81

On the map: LLM safety and security → Machine unlearning

Abstract (arXiv)

Machine unlearning is a promising approach to mitigate undesirable memorization of training data in ML models. However, in this work we show that existing approaches for unlearning in LLMs are surprisingly susceptible to a simple set of textitbenign relearning attacks. With access to only a small and potentially loosely related set of data, we find that we can ''jog'' the memory of unlearned models to reverse the effects of unlearning. For example, we show that relearning on public medical articles can lead an unlearned LLM to output harmful knowledge about bioweapons, and relearning general wiki information about the book series Harry Potter can force the model to output verbatim memorized text. We formalize this unlearning-relearning pipeline, explore the attack across three popular unlearning benchmarks, and discuss future directions and guidelines that result from our study. Our work indicates that current approximate unlearning methods simply suppress the model outputs and fail to robustly forget target knowledge in the LLMs.

Key facts. Cited by 81 papers in the map (data through 7 Oct 2026). Placed in Machine unlearning, one of 12 directions in LLM safety and security. Builds on 31 of its 44 references that are in the map. First posted on arXiv 19 Jun 2024.

Builds on

31 of 44 references.

Closest ancestors · 15

Same direction, newest first

  1. 2024RWKU: Benchmarking Real-World Knowledge Unlearning for Large Language Models16 Jun
  2. 2024Large Language Model Unlearning via Embedding-Corrupted Prompts12 Jun
  3. 2024SOUL: Unlocking the Power of Second-Order Optimization for LLM Unlearning28 Apr
  4. 2024Negative Preference Optimization: From Catastrophic Collapse to Effective Unlearning8 Apr
  5. 2024Guardrail Baselines for Unlearning in LLMs5 Mar
  6. 2024Eight Methods to Evaluate Robust Unlearning in LLMs26 Feb
  7. 2024Rethinking Machine Unlearning for Large Language Models13 Feb
  8. 2024TOFU: A Task of Fictitious Unlearning for LLMs11 Jan
  9. 2023Unlearn What You Want to Forget: Efficient Unlearning for LLMs31 Oct
  10. 2023Large Language Model Unlearning14 Oct
  11. 2023In-Context Unlearning: Language Models as Few Shot Unlearners11 Oct
  12. 2023Who's Harry Potter? Approximate Unlearning in LLMs3 Oct
3 more
  1. 2023Towards Unbounded Machine Unlearning20 Feb
  2. 2022Continual Learning and Private Unlearning24 Mar
  3. 2021Fast Yet Effective Machine Unlearning17 Nov

Other references · 16

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  4. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  5. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  6. 2020Extracting Training Data from Large Language Modelscited by 2,144
  7. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,106
  8. 2019Machine Unlearningcited by 1,020
  9. 2019Eternal Sunshine of the Spotless Net: Selective Forgetting in Deep Networkscited by 599
  10. 2023Textbooks Are All You Need II: phi-1.5 technical reportcited by 559
  11. 2023Zephyr: Direct Distillation of LM Alignmentcited by 498
  12. 2024The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearningcited by 488
4 more
  1. 2019Making AI Forget You: Data Deletion in Machine Learningcited by 447
  2. 2020Descent-to-Delete: Gradient-Based Methods for Machine Unlearningcited by 303
  3. 2023Can Sensitive Information Be Deleted From LLMs? Objectives for Defending Against Extraction Attackscited by 156
  4. 2024Large Language Models Relearn Removed Conceptscited by 26

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 30; 2026: 48.

81 papers.

Sort:

By area:

  1. 2024Simplicity Prevails: Rethinking Negative Preference Optimization for LLM Unlearningcited by 160
  2. 2025The Attacker Moves Second: Stronger Adaptive Attacks Bypass Defenses Against Llm Jailbreaks and Prompt Injectionscited by 108
  3. 2025OpenUnlearning: Accelerating LLM Unlearning via Unified Benchmarking of Methods and Metricscited by 80
  4. 2024On Large Language Model Continual Unlearningcited by 51
  5. 2025Unlearning Isn't Deletion: Investigating Reversibility of Machine Unlearning in LLMscited by 41
  6. 2025A Comprehensive Survey of Machine Unlearning Techniques for Large Language Modelscited by 30
  7. 2024Towards Robust Knowledge Unlearning: An Adversarial Framework for Assessing and Improving Unlearning Robustness in Large Language Modelscited by 24
  8. 2025Tool Unlearning for Tool-Augmented LLMscited by 17
  9. 2025Which Retain Set Matters for LLM Unlearning? A Case Study on Entity Unlearningcited by 14
  10. 2025Exact Unlearning of Finetuning Data via Model Merging at Scalecited by 14
  11. 2025BLUR: A Benchmark for LLM Unlearning Robust to Forget-Retain Overlapcited by 12
  12. 2025Precise In-Parameter Concept Erasure in Large Language Modelscited by 11
38 more
  1. 2025Distillation Robustifies Unlearningcited by 11
  2. 2026Shaping capabilities with token-level data filteringcited by 11
  3. 2025Constrained Entropic Unlearning: A Primal-Dual Framework for Large Language Modelscited by 10
  4. 2025Beyond Data Filtering: Knowledge Localization for Capability Removal in LLMscited by 8
  5. 2025AI Supply Chains: An Emerging Ecosystem of AI Actors, Products, and Servicescited by 7
  6. 2025Existing Large Language Model Unlearning Evaluations Are Inconclusivecited by 7
  7. 2025Improving LLM Unlearning Robustness via Random Perturbationscited by 6
  8. 2025Step-by-Step Reasoning Attack: Revealing 'Erased' Knowledge in Large Language Modelscited by 6
  9. 2025On the Impossibility of Retrain Equivalence in Machine Unlearningcited by 6
  10. 2026Attention Smoothing Is All You Need For Unlearningcited by 6
  11. 2025Unlearned but Not Forgotten: Data Extraction after Exact Unlearning in LLMcited by 5
  12. 2025Erase or Hide? Suppressing Spurious Unlearning Neurons for Robust Unlearningcited by 5
  13. 2026Is Gradient Ascent Really Necessary? Memorize to Forget for Machine Unlearningcited by 5
  14. 2025Forgetting to Forget: Attention Sink as A Gateway for Backdooring LLM Unlearningcited by 4
  15. 2026REBEL: Hidden Knowledge Recovery via Evolutionary-Based Evaluation Loopcited by 4
  16. 2025Unlearning's Blind Spots: Over-Unlearning and Prototypical Relearning Attackcited by 3
  17. 2025Understanding the Dilemma of Unlearning for Large Language Modelscited by 3
  18. 2026DUET: Distilled LLM Unlearning from an Efficiently Contextualized Teachercited by 3
  19. 2026Hair-Trigger Alignment: Black-Box Evaluation Cannot Guarantee Post-Update Alignmentcited by 3
  20. 2026Is your algorithm unlearning or untraining?cited by 3
  21. 2026Can an AI Assistant Really Forget? Auditable Deletion from Addressable Memorycited by 3
  22. 2025Align-then-Unlearn: Embedding Alignment for LLM Unlearningcited by 2
  23. 2025Memory Self-Regeneration: Uncovering Hidden Knowledge in Unlearned Modelscited by 2
  24. 2025Forget to Know, Remember to Use: Context-Aware Unlearning for Large Language Modelscited by 2
  25. 2026Multilingual Unlearning in LLMs: Transfer, Dynamics, and Reversibilitycited by 2
  26. 2026LACUNA: A Testbed for Evaluating Localization Precision for LLM Unlearningcited by 2
  27. 2025One-Point Contraction: Erasing Representational Separability toward Irreversible Deep Forgettingcited by 1
  28. 2025Beyond Sharp Minima: Robust LLM Unlearning via Feedback-Guided Multi-Point Optimizationcited by 1
  29. 2025Bits Leaked per Query: Information-Theoretic Bounds on Adversarial Attacks against LLMscited by 1
  30. 2025Robust MLLM Unlearning via Visual Knowledge Distillationcited by 1
  31. 2026Beyond Forgetting: Representation Misdirection Elicits Controllable Side Behaviors and Capabilitiescited by 1
  32. 2026Rethinking Benign Relearning: Syntax as the Hidden Driver of Unlearning Failurescited by 1
  33. 2026TamperBench: Systematically Stress-Testing LLM Safety Under Fine-Tuning and Tamperingcited by 1
  34. 2026Detoxifying LLMs via Representation Erasure-Based Preference Optimizationcited by 1
  35. 2026A Comprehensive Evaluation of LLM Unlearning Robustness under Multi-Turn Interactioncited by 1
  36. 2026Secure Forgetting: A Framework for Privacy-Driven Unlearning in Large Language Model (LLM)-Based Agentscited by 1
  37. 2026Robust LLM Unlearning Against Relearning Attacks: The Minor Components in Representations Mattercited by 1
  38. 2026GROM: Gradient-Free Rapid One-Shot Machine Unlearningcited by 1

50 most cited of 81 All 81 (JSON)