mlsanityData to 7 Oct 2026

To Each (Textual Sequence) Its Own: Improving Memorized-Data Unlearning in Large Language Models

arXiv 2405.03097 ↗First posted 6 May 2024cs.LG cs.AI cs.CLcited by 42

On the map: LLM safety and security → Machine unlearning

Abstract (arXiv)

LLMs have been found to memorize training textual sequences and regurgitate verbatim said sequences during text generation time. This fact is known to be the cause of privacy and related (e.g., copyright) problems. Unlearning in LLMs then takes the form of devising new algorithms that will properly deal with these side-effects of memorized data, while not hurting the model's utility. We offer a fresh perspective towards this goal, namely, that each textual sequence to be forgotten should be treated differently when being unlearned based on its degree of memorization within the LLM. We contribute a new metric for measuring unlearning quality, an adversarial attack showing that SOTA algorithms lacking this perspective fail for privacy, and two new unlearning methods based on Gradient Ascent and Task Arithmetic, respectively. A comprehensive performance evaluation across an extensive suite of NLP tasks then mapped the solution space, identifying the best solutions under different scales in model capacities and forget set sizes and quantified the gains of the new approaches.

Key facts. Cited by 42 papers in the map (data through 7 Oct 2026). Placed in Machine unlearning, one of 12 directions in LLM safety and security. Builds on 49 of its 65 references that are in the map. First posted on arXiv 6 May 2024.

Builds on

49 of 65 references.

Closest ancestors · 9

Same direction, newest first

  1. 2024TOFU: A Task of Fictitious Unlearning for LLMs11 Jan
  2. 2023Unlearn What You Want to Forget: Efficient Unlearning for LLMs31 Oct
  3. 2023In-Context Unlearning: Language Models as Few Shot Unlearners11 Oct
  4. 2023Towards Unbounded Machine Unlearning20 Feb
  5. 2022Knowledge Unlearning for Mitigating Privacy Risks in Language Models4 Oct
  6. 2022A Survey of Machine Unlearning6 Sep
  7. 2022Quark: Controllable Text Generation with Reinforced Unlearning26 May
  8. 2022Can Bad Teaching Induce Forgetting? Unlearning in Deep Networks using an Incompetent Teacher17 May
  9. 2021Unrolling SGD: Understanding Factors Influencing Machine Unlearning27 Sep

Other references · 40

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  3. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  4. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  5. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
  6. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,818
  7. 2016Deep Learning with Differential Privacycited by 2,682
  8. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  9. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,346
  10. 2020Extracting Training Data from Large Language Modelscited by 2,144
  11. 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,271
  12. 2022Editing Models with Task Arithmeticcited by 1,076
28 more
  1. 2016The LAMBADA dataset: Word prediction requiring a broad discourse contextcited by 960
  2. 2021Membership Inference Attacks From First Principlescited by 839
  3. 2022Quantifying Memorization Across Neural Language Modelscited by 755
  4. 2021Deduplicating Training Data Makes Language Models Bettercited by 705
  5. 2019MathQA: Towards Interpretable Math Word Problem Solving with Operation-Based Formalismscited by 653
  6. 2019Eternal Sunshine of the Spotless Net: Selective Forgetting in Deep Networkscited by 599
  7. 2018Wizard of Wikipedia: Knowledge-Powered Conversational agentscited by 530
  8. 2019SAMSum Corpus: A Human-annotated Dialogue Dataset for Abstractive Summarizationcited by 530
  9. 2019Does Learning Require Memorization? A Short Tale about a Long Tailcited by 452
  10. 2021Large Language Models Can Be Strong Differentially Private Learnerscited by 405
  11. 2021Differentially Private Fine-tuning of Language Modelscited by 392
  12. 2020Amnesiac Machine Learningcited by 305
  13. 2023Membership Inference Attacks against Language Models via Neighbourhood Comparisoncited by 249
  14. 2022Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language Modelscited by 218
  15. 2021Internet-Augmented Dialogue Generationcited by 217
  16. 2022What Does it Mean for a Language Model to Preserve Privacy?cited by 194
  17. 2023Knowledge Editing for Large Language Models: A Surveycited by 189
  18. 2024A Comprehensive Study of Knowledge Editing for Large Language Modelscited by 157
  19. 2021Counterfactual Memorization in Neural Language Modelscited by 155
  20. 2020NeuroLogic Decoding: (Un)supervised Neural Text Generation with Predicate Logic Constraintscited by 146
  21. 2020Can You Put it All Together: Evaluating Conversational Agents' Ability to Blend Skillscited by 130
  22. 2022Measuring Forgetting of Memorized Training Examplescited by 118
  23. 2021Large-Scale Differentially Private BERTcited by 117
  24. 2023Privacy Issues in Large Language Models: A Surveycited by 57
  25. 2022Differentially Private Decoding in Large Language Modelscited by 34
  26. 2023Machine Unlearning in Learned Databases: An Experimental Analysiscited by 17
  27. 2018Leveraging Random Label Memorization for Unsupervised Pre-Trainingcited by 12
  28. 2023Forgetting Private Textual Sequences in Language Models via Leave-One-Out Ensemblecited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 12; 2025: 18; 2026: 12.

42 papers.

Sort:

By area:

  1. 2024What makes unlearning hard and what to do about itcited by 89
  2. 2024Catastrophic Failure of LLM Unlearning via Quantizationcited by 76
  3. 2024Position: LLM Unlearning Benchmarks are Weak Measures of Progresscited by 58
  4. 2024Revisiting Who's Harry Potter: Towards Targeted Unlearning from a Causal Intervention Perspectivecited by 33
  5. 2024Towards Effective Evaluations and Comparisons for LLM Unlearning Methodscited by 27
  6. 2025Invariance Makes LLM Unlearning Resilient Even to Unanticipated Downstream Fine-Tuningcited by 20
  7. 2025Tool Unlearning for Tool-Augmented LLMscited by 17
  8. 2024MU-Bench: A Multitask Multimodal Benchmark for Machine Unlearningcited by 16
  9. 2024UnStar: Unlearning with Self-Taught Anti-Sample Reasoning for LLMscited by 14
  10. 2025Ascent Fails to Forgetcited by 12
  11. 2024Mitigating Memorization In Language Modelscited by 11
  12. 2025GRU: Mitigating the Trade-off between Unlearning and Retention for LLMscited by 11
30 more
  1. 2025LLM Unlearning Reveals a Stronger-Than-Expected Coreset Effect in Current Benchmarkscited by 11
  2. 2025Distillation Robustifies Unlearningcited by 11
  3. 2025SoK: Machine Unlearning for Large Language Modelscited by 9
  4. 2025Operationalizing Data Minimization for Privacy-Preserving LLM Promptingcited by 8
  5. 2025Memorization Sinks: Isolating Memorization during LLM Trainingcited by 7
  6. 2024Scalability of memorization-based machine unlearningcited by 5
  7. 2025Erase or Hide? Suppressing Spurious Unlearning Neurons for Robust Unlearningcited by 5
  8. 2024Unlearning in- vs. out-of-distribution data in LLMs under gradient-based methodcited by 4
  9. 2025DUSK: Do Not Unlearn Shared Knowledgecited by 4
  10. 2025Leveraging Per-Instance Privacy for Machine Unlearningcited by 4
  11. 2026REBEL: Hidden Knowledge Recovery via Evolutionary-Based Evaluation Loopcited by 4
  12. 2025LoReUn: Data Itself Implicitly Provides Cues to Improve Machine Unlearningcited by 3
  13. 2025The Erasure Illusion: Stress-Testing the Generalization of LLM Forgetting Evaluationcited by 3
  14. 2026Is your algorithm unlearning or untraining?cited by 3
  15. 2024RESTOR: Knowledge Recovery in Machine Unlearningcited by 2
  16. 2026GSS: Gated Subspace Steering for Selective Memorization Mitigation in LLMscited by 2
  17. 2025A Lightweight Method to Disrupt Memorized Sequences in LLMcited by 1
  18. 2025MIAT: Maneuver-Intention-Aware Transformer for Spatio-Temporal Trajectory Predictioncited by 1
  19. 2025Learning-Time Encoding Shapes Unlearning in LLMscited by 1
  20. 2026Rethinking Benign Relearning: Syntax as the Hidden Driver of Unlearning Failurescited by 1
  21. 2024Deep Model Merging: The Sister of Neural Network Interpretability -- A Surveycited by 0
  22. 2025Understanding Machine Unlearning Through the Lens of Mode Connectivitycited by 0
  23. 2026Forgetting Similar Samples: Can Machine Unlearning Do it Better?cited by 0
  24. 2026SAU: Sparsity-Aware Unlearning for LLMs via Gradient Masking and Importance Redistributioncited by 0
  25. 2026EvoMU: Evolutionary Machine Unlearningcited by 0
  26. 2026Benchmarking Unlearning for Vision Transformerscited by 0
  27. 2026You Don't Need All That Attention: Surgical Memorization Mitigation in Text-to-Image Diffusion Modelscited by 0
  28. 2026Understanding Machine Unlearning Through the Lens of Mode Connectivitycited by 0
  29. 2026PRMU: A Corpus-Free Benchmark for Person-Centric Knowledge Unlearning in Multimodal Large Language Modelscited by 0
  30. 2026The More Popular, The Harder to Forget: Adaptive Popularity for LLM Unlearningcited by 0