mlsanityData to 7 Oct 2026

OBLIVIATE: Robust and Practical Machine Unlearning for Large Language Models

arXiv 2505.04416 ↗First posted 7 May 2025cs.CL cs.AI cs.CRcited by 9

On the map: LLM safety and security → Machine unlearning

Abstract (arXiv)

Large language models (LLMs) trained over extensive corpora risk memorizing sensitive, copyrighted, or toxic content. To address this, we propose OBLIVIATE, a robust unlearning framework that removes targeted data while preserving model utility. The framework follows a structured process: extracting target tokens, building retain sets, and fine-tuning with a tailored loss function comprising three components -- masking, distillation, and world fact. Using low-rank adapters (LoRA) ensures efficiency without compromising unlearning quality. We conduct experiments on multiple datasets, including Harry Potter series, WMDP, and TOFU, using a comprehensive suite of metrics: forget quality (via a new document-level memorization score), model utility, and fluency. Results demonstrate its effectiveness in resisting membership inference attacks, minimizing the impact on retained data, and maintaining robustness across diverse scenarios.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Placed in Machine unlearning, one of 12 directions in LLM safety and security. Builds on 41 of its 54 references that are in the map. First posted on arXiv 7 May 2025.

Builds on

41 of 54 references.

Closest ancestors · 16

Same direction, newest first

  1. 2025FUNU: Boosting Machine Unlearning Efficiency by Filtering Unnecessary Unlearning28 Jan
  2. 2024Position: LLM Unlearning Benchmarks are Weak Measures of Progress3 Oct
  3. 2024Erasing Conceptual Knowledge from Language Models3 Oct
  4. 2024MEOW: MEMOry Supervised LLM Unlearning Via Inverted Facts18 Sep
  5. 2024Revisiting Who's Harry Potter: Towards Targeted Unlearning from a Causal Intervention Perspective24 Jul
  6. 2024Protecting Privacy Through Approximating Optimal Parameters for Sequence Unlearning in Language Models20 Jun
  7. 2024Reversing the Forget-Retain Objectives: An Efficient LLM Unlearning Framework from Logit Difference12 Jun
  8. 2024Large Language Model Unlearning via Embedding-Corrupted Prompts12 Jun
  9. 2024Negative Preference Optimization: From Catastrophic Collapse to Effective Unlearning8 Apr
  10. 2024TOFU: A Task of Fictitious Unlearning for LLMs11 Jan
  11. 2023Breaking the Trilemma of Privacy, Utility, Efficiency via Controllable Machine Unlearning28 Oct
  12. 2023In-Context Unlearning: Language Models as Few Shot Unlearners11 Oct
4 more
  1. 2023Who's Harry Potter? Approximate Unlearning in LLMs3 Oct
  2. 2022Knowledge Unlearning for Mitigating Privacy Risks in Language Models4 Oct
  3. 2021Adaptive Machine Unlearning8 Jun
  4. 2021Remember What You Want to Forget: Algorithms for Machine Unlearning4 Mar

Other references · 25

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  4. 2016Pointer Sentinel Mixture Modelscited by 2,992
  5. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  6. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  7. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,221
  8. 2020Extracting Training Data from Large Language Modelscited by 2,144
  9. 2019Language Models as Knowledge Bases?cited by 1,922
  10. 2019Machine Unlearningcited by 1,020
  11. 2019Eternal Sunshine of the Spotless Net: Selective Forgetting in Deep Networkscited by 599
  12. 2023Dissecting Recall of Factual Associations in Auto-Regressive Language Modelscited by 512
13 more
  1. 2023Zephyr: Direct Distillation of LM Alignmentcited by 498
  2. 2024The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearningcited by 488
  3. 2019Making AI Forget You: Data Deletion in Machine Learningcited by 447
  4. 2023Scalable Extraction of Training Data from (Production) Language Modelscited by 415
  5. 2020Descent-to-Delete: Gradient-Based Methods for Machine Unlearningcited by 303
  6. 2022Estimating the Carbon Footprint of BLOOM, a 176B Parameter Language Modelcited by 264
  7. 2021Towards Continual Knowledge Learning of Language Modelscited by 172
  8. 2020The Lipschitz Constant of Self-Attentioncited by 171
  9. 2024On Memorization of Large Language Models in Logical Reasoningcited by 114
  10. 2023DP-Forward: Fine-tuning and Inference on Language Models with Differential Privacy in Forward Passcited by 65
  11. 2024Membership Inference Attacks and Defenses in Federated Learning: A Surveycited by 27
  12. 2024Large Language Models Relearn Removed Conceptscited by 26
  13. 2021Recent Trends in Named Entity Recognition (NER)cited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 6.

9 papers.

  1. 2025Unlearning Isn't Deletion: Investigating Reversibility of Machine Unlearning in LLMscited by 41
  2. 2026REBEL: Hidden Knowledge Recovery via Evolutionary-Based Evaluation Loopcited by 4
  3. 2026STaR: Sensitive Trajectory Regulation for Unlearning in Large Reasoning Modelscited by 2
  4. 2025Beyond Sharp Minima: Robust LLM Unlearning via Feedback-Guided Multi-Point Optimizationcited by 1
  5. 2025RapidUn: Influence-Driven Parameter Reweighting for Efficient Large Language Model Unlearningcited by 0
  6. 2026FIT to Forget: Robust Continual Unlearning for Large Language Modelscited by 0
  7. 2026Machine Unlearning in Low-Dimensional Feature Subspacecited by 0
  8. 2026EvoMU: Evolutionary Machine Unlearningcited by 0
  9. 2026BLADE: Bilevel Low-rank Augmented-Lagrangian Erasure for LLM Unlearningcited by 0