mlsanityData to 7 Oct 2026

Unlearning in- vs. out-of-distribution data in LLMs under gradient-based method

arXiv 2411.04388 ↗First posted 7 Nov 2024cs.LGcited by 4

On the map: LLM safety and security → Machine unlearning

Abstract (arXiv)

Machine unlearning aims to solve the problem of removing the influence of selected training examples from a learned model. Despite the increasing attention to this problem, it remains an open research question how to evaluate unlearning in large language models (LLMs), and what are the critical properties of the data to be unlearned that affect the quality and efficiency of unlearning. This work formalizes a metric to evaluate unlearning quality in generative models, and uses it to assess the trade-offs between unlearning quality and performance. We demonstrate that unlearning out-of-distribution examples requires more unlearning steps but overall presents a better trade-off overall. For in-distribution examples, however, we observe a rapid decay in performance as unlearning progresses. We further evaluate how example's memorization and difficulty affect unlearning under a classical gradient ascent-based approach.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in Machine unlearning, one of 12 directions in LLM safety and security. Builds on 32 of its 38 references that are in the map. First posted on arXiv 7 Nov 2024.

Builds on

32 of 38 references.

Closest ancestors · 9

Same direction, newest first

  1. 2024MUSE: Machine Unlearning Six-Way Evaluation for Language Models8 Jul
  2. 2024Are we making progress in unlearning? Findings from the first NeurIPS unlearning competition13 Jun
  3. 2024To Each (Textual Sequence) Its Own: Improving Memorized-Data Unlearning in Large Language Models6 May
  4. 2024Inexact Unlearning Needs More Careful Evaluations to Avoid a False Sense of Privacy2 Mar
  5. 2024Eight Methods to Evaluate Robust Unlearning in LLMs26 Feb
  6. 2024TOFU: A Task of Fictitious Unlearning for LLMs11 Jan
  7. 2022Knowledge Unlearning for Mitigating Privacy Risks in Language Models4 Oct
  8. 2021Adaptive Machine Unlearning8 Jun
  9. 2021Remember What You Want to Forget: Algorithms for Machine Unlearning4 Mar

Other references · 23

  1. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  2. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,221
  3. 2020Extracting Training Data from Large Language Modelscited by 2,144
  4. 2022Mass-Editing Memory in a Transformercited by 976
  5. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 847
  6. 2021Membership Inference Attacks From First Principlescited by 839
  7. 2022Quantifying Memorization Across Neural Language Modelscited by 755
  8. 2021Deduplicating Training Data Makes Language Models Bettercited by 705
  9. 2018ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Modelscited by 475
  10. 2019Does Learning Require Memorization? A Short Tale about a Long Tailcited by 452
  11. 2020Label-Only Membership Inference Attackscited by 371
  12. 2020Descent-to-Delete: Gradient-Based Methods for Machine Unlearningcited by 303
11 more
  1. 2013The Composition Theorem for Differential Privacycited by 290
  2. 2023Analyzing Leakage of Personally Identifiable Information in Language Modelscited by 278
  3. 2023Membership Inference Attacks against Language Models via Neighbourhood Comparisoncited by 249
  4. 2020Auditing Differentially Private Machine Learning: How Private is Private SGD?cited by 213
  5. 2022Scaling Up Models and Data with textttt5x and textttseqiocited by 187
  6. 2022Quantifying Privacy Risks of Masked Language Models Using Membership Inference Attackscited by 168
  7. 2023Can Sensitive Information Be Deleted From LLMs? Objectives for Defending Against Extraction Attackscited by 156
  8. 2021Does BERT Pretrained on Clinical Notes Reveal Sensitive Data?cited by 91
  9. 2018Comprehensive Privacy Analysis of Deep Learning: Passive and Active White-box Inference Attacks against Centralized and Federated Learningcited by 75
  10. 2020KART: Parameterization of Privacy Leakage Scenarios from Pre-trained Language Modelscited by 7
  11. 2023A Note On Interpreting Canary Exposurecited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 1.

4 papers.

  1. 2025ReLearn: Unlearning via Learning for Large Language Modelscited by 20
  2. 2025Leveraging Per-Instance Privacy for Machine Unlearningcited by 4
  3. 2025MIAT: Maneuver-Intention-Aware Transformer for Spatio-Temporal Trajectory Predictioncited by 1
  4. 2026The More Popular, The Harder to Forget: Adaptive Popularity for LLM Unlearningcited by 0