mlsanityData to 7 Oct 2026

Unveiling Entity-Level Unlearning for Large Language Models: A Comprehensive Analysis

arXiv 2406.15796 ↗First posted 22 Jun 2024cs.CLcited by 14

On the map: LLM safety and security → Machine unlearning

Abstract (arXiv)

Large language model unlearning has garnered increasing attention due to its potential to address security and privacy concerns, leading to extensive research in the field. However, much of this research has concentrated on instance-level unlearning, specifically targeting the removal of predefined instances containing sensitive content. This focus has left a significant gap in the exploration of full entity-level unlearning, which is critical in real-world scenarios such as copyright protection. To this end, we propose a novel task of Entity-level unlearning, which aims to erase entity-related knowledge from the target model completely. To thoroughly investigate this task, we systematically evaluate trending unlearning algorithms, revealing that current methods struggle to achieve effective entity-level unlearning. Then, we further explore the factors that influence the performance of the unlearning algorithms, identifying that knowledge coverage and the size of the forget set play pivotal roles. Notably, our analysis also uncovers that entities introduced through fine-tuning are more vulnerable to unlearning than pre-trained entities. These findings collectively offer valuable insights for advancing entity-level unlearning for LLMs.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Placed in Machine unlearning, one of 12 directions in LLM safety and security. Builds on 29 of its 48 references that are in the map. First posted on arXiv 22 Jun 2024.

Builds on

29 of 48 references.

Closest ancestors · 8

Same direction, newest first

  1. 2024Negative Preference Optimization: From Catastrophic Collapse to Effective Unlearning8 Apr
  2. 2024Eight Methods to Evaluate Robust Unlearning in LLMs26 Feb
  3. 2024Towards Safer Large Language Models through Machine Unlearning15 Feb
  4. 2024TOFU: A Task of Fictitious Unlearning for LLMs11 Jan
  5. 2023Unlearn What You Want to Forget: Efficient Unlearning for LLMs31 Oct
  6. 2023Model Sparsity Can Simplify Machine Unlearning11 Apr
  7. 2022Knowledge Unlearning for Mitigating Privacy Risks in Language Models4 Oct
  8. 2022Quark: Controllable Text Generation with Reinforced Unlearning26 May

Other references · 21

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  5. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  6. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  7. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  8. 2023BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Datasetcited by 828
  9. 2023Textbooks Are All You Need II: phi-1.5 technical reportcited by 559
  10. 2024The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearningcited by 488
  11. 2023Editing Large Language Models: Problems, Methods, and Opportunitiescited by 367
  12. 2024Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations?cited by 227
9 more
  1. 2024Security and Privacy Challenges of Large Language Models: A Surveycited by 179
  2. 2023Can Sensitive Information Be Deleted From LLMs? Objectives for Defending Against Extraction Attackscited by 156
  3. 2024Language Models are Homer Simpson! Safety Re-Alignment of Fine-tuned Language Models through Task Arithmeticcited by 105
  4. 2023Identifying and Mitigating the Security Risks of Generative AIcited by 90
  5. 2024Detoxifying Large Language Models via Knowledge Editingcited by 86
  6. 2024Eraser: Jailbreaking Defense in Large Language Models via Unlearning Harmful Knowledgecited by 48
  7. 2023"According to ...": Prompting Language Models Improves Quoting from Pre-Training Datacited by 42
  8. 2023Trends in Integration of Knowledge and Large Language Models: A Survey and Taxonomy of Methods, Benchmarks, and Applicationscited by 28
  9. 2024Do LLMs Find Human Answers To Fact-Driven Questions Perplexing? A Case Study on Redditcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 9; 2026: 5.

14 papers.

Sort:

By area:

  1. 2025ReLearn: Unlearning via Learning for Large Language Modelscited by 20
  2. 2025Do LLMs Really Forget? Evaluating Unlearning with Knowledge Correlation and Confidence Awarenesscited by 13
  3. 2025SAUCE: Selective Concept Unlearning in Vision-Language Models with Sparse Autoencoderscited by 12
  4. 2025Not Every Token Needs Forgetting: Selective Unlearning to Limit Change in Utility in Large Language Model Unlearningcited by 7
  5. 2025Unilogit: Robust Machine Unlearning for LLMs Using Uniform-Target Self-Distillationcited by 4
  6. 2025ZJUKLAB at SemEval-2025 Task 4: Unlearning via Model Mergingcited by 2
  7. 2025A Survey on Unlearning in Large Language Modelscited by 2
  8. 2025FAME: Fictional Actors for Multilingual Erasurecited by 1
  9. 2026From Anchors to Supervision: Memory-Graph Guided Corpus-Free Unlearning for Large Language Modelscited by 1
  10. 2026Does Forgetting Transfer Across Modalities? A Real-World Benchmark for Cross-Modal Knowledge Unlearning Evaluationcited by 1
  11. 2026UnPII: Unlearning Personally Identifiable Information with Quantifiable Exposure Riskcited by 0
  12. 2025Consistency-Aware Editing for Entity-level Unlearning in Language Modelscited by 0
2 more
  1. 2026Representation-Aware Unlearning via Activation Signatures: From Suppression to Entity-Signature Erasurecited by 0
  2. 2026Making LLMs Truly Forget: Deep Unlearning by Searching, Selecting, and Severing Knowledge Pathscited by 0