mlsanityData to 14 Oct 2026

ReLearn: Unlearning via Learning for Large Language Models

arXiv 2502.11190 ↗First posted 16 Feb 2025cs.CL cs.AI cs.CVcited by 22

On the map: LLM safety and security → Machine unlearning

Abstract (arXiv)

Current unlearning methods for large language models usually rely on reverse optimization to reduce target token probabilities. However, this paradigm disrupts the subsequent tokens prediction, degrading model performance and linguistic coherence. Moreover, existing evaluation metrics overemphasize contextual forgetting while inadequately assessing response fluency and relevance. To address these challenges, we propose ReLearn, a data augmentation and fine-tuning pipeline for effective unlearning, along with a comprehensive evaluation framework. This framework introduces Knowledge Forgetting Rate (KFR) and Knowledge Retention Rate (KRR) to measure knowledge-level preservation, and Linguistic Score (LS) to evaluate generation quality. Our experiments show that ReLearn successfully achieves targeted forgetting while preserving high-quality output. Through mechanistic analysis, we further demonstrate how reverse optimization disrupts coherent text generation, while ReLearn preserves this essential capability. Code is available at https://github.com/zjunlp/unlearn.

Key facts. Cited by 22 papers in the map (data through 14 Oct 2026). Placed in Machine unlearning, one of 12 directions in LLM safety and security. Builds on 44 of its 75 references that are in the map. First posted on arXiv 16 Feb 2025.

Builds on

44 of 75 references.

Closest ancestors · 23

Same direction, newest first

  1. 2025Knowledge Swapping via Learning and Unlearning12 Feb
  2. 2025Open Problems in Machine Unlearning for AI Safety9 Jan
  3. 2024Underestimated Privacy Risks for Minority Populations in Large Language Model Unlearning11 Dec
  4. 2024Unified Parameter-Efficient Unlearning for LLMs30 Nov
  5. 2024Towards Robust Evaluation of Unlearning in LLMs via Data Transformations23 Nov
  6. 2024Unlearning in- vs. out-of-distribution data in LLMs under gradient-based method7 Nov
  7. 2024Benchmarking Vision Language Model Unlearning via Fictitious Facial Identity Dataset5 Nov
  8. 2024UnStar: Unlearning with Self-Taught Anti-Sample Reasoning for LLMs22 Oct
  9. 2024Position: LLM Unlearning Benchmarks are Weak Measures of Progress3 Oct
  10. 2024Erasing Conceptual Knowledge from Language Models3 Oct
  11. 2024MEOW: MEMOry Supervised LLM Unlearning Via Inverted Facts18 Sep
  12. 2024On the Limitations and Prospects of Machine Unlearning for Generative AI1 Aug
11 more
  1. 2024Learning to Refuse: Towards Mitigating Privacy Risks in LLMs14 Jul
  2. 2024MUSE: Machine Unlearning Six-Way Evaluation for Language Models8 Jul
  3. 2024Unveiling Entity-Level Unlearning for Large Language Models: A Comprehensive Analysis22 Jun
  4. 2024Negative Preference Optimization: From Catastrophic Collapse to Effective Unlearning8 Apr
  5. 2024Towards Safer Large Language Models through Machine Unlearning15 Feb
  6. 2024TOFU: A Task of Fictitious Unlearning for LLMs11 Jan
  7. 2023Unlearn What You Want to Forget: Efficient Unlearning for LLMs31 Oct
  8. 2023In-Context Unlearning: Language Models as Few Shot Unlearners11 Oct
  9. 2023Who's Harry Potter? Approximate Unlearning in LLMs3 Oct
  10. 2022Knowledge Unlearning for Mitigating Privacy Risks in Language Models4 Oct
  11. 2022Quark: Controllable Text Generation with Reinforced Unlearning26 May

Other references · 21

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  4. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  5. 2020Extracting Training Data from Large Language Modelscited by 2,189
  6. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,998
  7. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,651
  8. 2023Eliciting Latent Predictions from Transformers with the Tuned Lenscited by 623
  9. 2022Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Spacecited by 532
  10. 2023Dissecting Recall of Factual Associations in Auto-Regressive Language Modelscited by 528
  11. 2024The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearningcited by 509
  12. 2021Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Surveycited by 463
9 more
  1. 2022ProsocialDialog: A Prosocial Backbone for Conversational Agentscited by 109
  2. 2023The Earth is Flat because...: Investigating LLMs' Belief towards Misinformation via Persuasive Conversationcited by 109
  3. 2023Dr.ICL: Demonstration-Retrieved In-context Learningcited by 72
  4. 2023Evaluation Metrics in the Era of GPT-4: Reliably Evaluating Large Language Models on Sequence to Sequence Taskscited by 53
  5. 2024Eraser: Jailbreaking Defense in Large Language Models via Unlearning Harmful Knowledgecited by 51
  6. 2024Who's asking? User personas and the mechanics of latent misalignmentcited by 40
  7. 2024LM Transparency Tool: Interactive Tool for Analyzing Transformer Language Modelscited by 16
  8. 2025Analyzing Memorization in Large Language Models through the Lens of Model Attributioncited by 14
  9. 2024Detecting Training Data of Large Language Models via Expectation Maximizationcited by 13

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 11; 2026: 11.

22 papers.

Sort:

By area:

  1. 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 116
  2. 2025OpenUnlearning: Accelerating LLM Unlearning via Unified Benchmarking of Methods and Metricscited by 83
  3. 2025Model Unlearning via Sparse Autoencoder Subspace Guided Projectionscited by 18
  4. 2025SoK: Machine Unlearning for Large Language Modelscited by 11
  5. 2025MLLMEraser: Achieving Test-Time Unlearning in Multimodal Large Language Models through Activation Steeringcited by 10
  6. 2025OFFSIDE: Benchmarking Unlearning Misinformation in Multimodal Large Language Modelscited by 8
  7. 2025DRAGON: Guard LLM Unlearning in Context via Negative Detection and Reasoningcited by 7
  8. 2025A Survey on Unlearning in Large Language Modelscited by 4
  9. 2026REBEL: Hidden Knowledge Recovery via Evolutionary-Based Evaluation Loopcited by 4
  10. 2025ContinualFlow: Learning and Unlearning with Neural Flow Matchingcited by 3
  11. 2025KnowledgeSmith: Uncovering Knowledge Updating in LLMs with Model Editing and Unlearningcited by 3
  12. 2026Simulating Novice Students Using Machine Unlearning and Relearning in Large Language Modelscited by 3
10 more
  1. 2026Steerability of Instrumental-Convergence Tendencies in LLMscited by 1
  2. 2026From Anchors to Supervision: Memory-Graph Guided Corpus-Free Unlearning for Large Language Modelscited by 1
  3. 2025RapidUn: Influence-Driven Parameter Reweighting for Efficient Large Language Model Unlearningcited by 0
  4. 2026EvoMU: Evolutionary Machine Unlearningcited by 0
  5. 2026Retain-Neutral Surrogates for Min-Max Unlearningcited by 0
  6. 2026SPACE: Source-free Proxy Anchor Concept Erasure for MLLMscited by 0
  7. 2026Null-Space Constrained Low-Rank Adaptation for Response-Specified Large Language Model Unlearningcited by 0
  8. 2026Test-Time Unlearning via Sparse Autoencodercited by 0
  9. 2026Faithful Dual-constrained Erasure for Robust LLM Safety Alignmentcited by 0
  10. 2026CUSE: Fine-Grained Multimodal Unlearning with Counterfactual Guidance and On-Policy Self-Distillationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Unlearning That Lasts: Utility-Preserving, Robust, and Almost Irreversible Forgetting in LLMs
  2. 2025Forget to Know, Remember to Use: Context-Aware Unlearning for Large Language Models
  3. 2026Not Every Change Is Necessary: Recoverable Drift in Large Language Model Unlearning
  4. 2025Unlearning Isn't Deletion: Investigating Reversibility of Machine Unlearning in LLMs
  5. 2026UnlearningSoup: Is Repeated Tuning Necessary for Large Language Model Unlearning?
  6. 2025Not Every Token Needs Forgetting: Selective Unlearning to Limit Change in Utility in Large Language Model Unlearning
  7. 2026Learning What to Forget: Improving LLM Unlearning via Learned Token-Level Importance
  8. 2025BLUR: A Benchmark for LLM Unlearning Robust to Forget-Retain Overlap