mlsanityData to 7 Oct 2026

On the Limitations and Prospects of Machine Unlearning for Generative AI

arXiv 2408.00376 ↗First posted 1 Aug 2024cs.LG cs.AIcited by 12

On the map: LLM safety and security → Machine unlearning

Abstract (arXiv)

Generative AI (GenAI), which aims to synthesize realistic and diverse data samples from latent variables or other data modalities, has achieved remarkable results in various domains, such as natural language, images, audio, and graphs. However, they also pose challenges and risks to data privacy, security, and ethics. Machine unlearning is the process of removing or weakening the influence of specific data samples or features from a trained model, without affecting its performance on other data or tasks. While machine unlearning has shown significant efficacy in traditional machine learning tasks, it is still unclear if it could help GenAI become safer and aligned with human desire. To this end, this position paper provides an in-depth discussion of the machine unlearning approaches for GenAI. Firstly, we formulate the problem of machine unlearning tasks on GenAI and introduce the background. Subsequently, we systematically examine the limitations of machine unlearning on GenAI models by focusing on the two representative branches: LLMs and image generative (diffusion) models. Finally, we provide our prospects mainly from three aspects: benchmark, evaluation metrics, and utility-unlearning trade-off, and conscientiously advocate for the future development of this field.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Placed in Machine unlearning, one of 12 directions in LLM safety and security. Builds on 38 of its 46 references that are in the map. First posted on arXiv 1 Aug 2024.

Builds on

38 of 46 references.

Closest ancestors · 11

Same direction, newest first

  1. 2024TOFU: A Task of Fictitious Unlearning for LLMs11 Jan
  2. 2023Knowledge Unlearning for LLMs: Tasks, Methods, and Challenges27 Nov
  3. 2023Unlearn What You Want to Forget: Efficient Unlearning for LLMs31 Oct
  4. 2023SalUn: Empowering Machine Unlearning via Gradient-based Weight Saliency in Both Image Classification and Generation19 Oct
  5. 2023Large Language Model Unlearning14 Oct
  6. 2023In-Context Unlearning: Language Models as Few Shot Unlearners11 Oct
  7. 2023Who's Harry Potter? Approximate Unlearning in LLMs3 Oct
  8. 2023KGA: A General Machine Unlearning Framework Based on Knowledge Gap Alignment11 May
  9. 2023Towards Unbounded Machine Unlearning20 Feb
  10. 2022Knowledge Unlearning for Mitigating Privacy Risks in Language Models4 Oct
  11. 2021Fast Yet Effective Machine Unlearning17 Nov

Other references · 27

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  4. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  5. 2020Gradient Surgery for Multi-Task Learningcited by 1,210
  6. 2022Editing Models with Task Arithmeticcited by 1,076
  7. 2019Machine Unlearningcited by 1,020
  8. 2021Membership Inference Attacks From First Principlescited by 839
  9. 2023Extracting Training Data from Diffusion Modelscited by 723
  10. 2023An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuningcited by 622
  11. 2023Erasing Concepts from Diffusion Modelscited by 530
  12. 2023Unified Concept Editing in Diffusion Modelscited by 377
15 more
  1. 2023A Survey of Hallucination in Large Foundation Modelscited by 365
  2. 2022Prompt-aligned Gradient for Prompt Tuningcited by 339
  3. 2023Ablating Concepts in Text-to-Image Diffusion Modelscited by 312
  4. 2023Forget-Me-Not: Learning to Forget in Text-to-Image Diffusion Modelscited by 297
  5. 2023Selective Amnesia: A Continual Learning Approach to Forgetting in Deep Generative Modelscited by 175
  6. 2022A Review on Language Models as Knowledge Basescited by 156
  7. 2023Investigating the Catastrophic Forgetting in Multimodal Large Language Modelscited by 119
  8. 2023Membership Inference Attacks against Diffusion Modelscited by 84
  9. 2024Unlearning Concepts in Diffusion Model via Concept Domain Correction and Concept Preserving Gradientcited by 69
  10. 2022Membership Inference Attacks Against Text-to-image Generation Modelscited by 66
  11. 2024Erasing Undesirable Influence in Diffusion Modelscited by 52
  12. 2022A survey of multimodal deep generative modelscited by 47
  13. 2024Machine Unlearning for Image-to-Image Generative Modelscited by 47
  14. 2023Towards More Realistic Membership Inference Attacks on Large Diffusion Modelscited by 41
  15. 2023Gradient Surgery for One-shot Unlearning on Generative Modelcited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 10; 2026: 2.

12 papers.

  1. 2025ReLearn: Unlearning via Learning for Large Language Modelscited by 20
  2. 2026Maximizing Local Entropy Where It Matters: Prefix-Aware Localized LLM Unlearningcited by 10
  3. 2025Position: Bridge the Gaps between Machine Unlearning and AI Regulationcited by 5
  4. 2025Efficient Utility-Preserving Machine Unlearning with Implicit Gradient Surgerycited by 5
  5. 2025Exploring Incremental Unlearning: Techniques, Challenges, and Future Directionscited by 2
  6. 2025ZJUKLAB at SemEval-2025 Task 4: Unlearning via Model Mergingcited by 2
  7. 2025Direct Token Optimization: A Self-contained Approach to Large Language Model Unlearningcited by 2
  8. 2025A Survey on Unlearning in Large Language Modelscited by 2
  9. 2025Selective, Controlled and Domain-Agnostic Unlearning in Pretrained CLIP: A Training- and Data-Free Approachcited by 2
  10. 2025Erasing CLIP Memories: Non-Destructive, Data-Free Zero-Shot class Unlearning in CLIP Modelscited by 1
  11. 2025GIFT: Gradient-aware Immunization of diffusion models against malicious Fine-Tuning with safe concepts retentioncited by 0
  12. 2026Shadow Unlearning: A Neuro-Semantic Approach to Fidelity-Preserving Faceless Forgetting in LLMscited by 0