# Right to be Forgotten in the Era of Large Language Models: Implications, Challenges, and Solutions

arXiv: https://arxiv.org/abs/2307.03941 · HTML: https://mlsanity.com/paper/2307.03941/

## Key facts

- Cited by 60 papers in the map (data through 14 Oct 2026).
- Placed in Machine unlearning, one of 12 directions in LLM safety and security.
- Builds on 22 of its 45 references that are in the map.
- First posted on arXiv 8 Jul 2023.

## Placed in

- LLM safety and security / [Machine unlearning](https://mlsanity.com/safety/machine-unlearning/)

## Builds on (most cited first, 22 of 22)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [A Survey of Large Language Models](https://mlsanity.com/paper/2303.18223/) (2023)
- [Emergent Abilities of Large Language Models](https://mlsanity.com/paper/2206.07682/) (2022)
- [Extracting Training Data from Large Language Models](https://mlsanity.com/paper/2012.07805/) (2020)
- [Machine Unlearning](https://mlsanity.com/paper/1912.03817/) (2019)
- [On Faithfulness and Factuality in Abstractive Summarization](https://mlsanity.com/paper/2005.00661/) (2020)
- [Extracting Training Data from Diffusion Models](https://mlsanity.com/paper/2301.13188/) (2023)
- [Eternal Sunshine of the Spotless Net: Selective Forgetting in Deep Networks](https://mlsanity.com/paper/1911.04933/) (2019)
- [Certified Data Removal from Machine Learning Models](https://mlsanity.com/paper/1911.03030/) (2019)
- [Fast Model Editing at Scale](https://mlsanity.com/paper/2110.11309/) (2021)
- [Memory-Based Model Editing at Scale](https://mlsanity.com/paper/2206.06520/) (2022)
- [Amnesiac Machine Learning](https://mlsanity.com/paper/2010.10981/) (2020)
- [Evaluating Verifiability in Generative Search Engines](https://mlsanity.com/paper/2304.09848/) (2023)
- [BERT has a Mouth, and It Must Speak: BERT as a Markov Random Field Language Model](https://mlsanity.com/paper/1902.04094/) (2019)
- [Synthetic Text Generation with Differential Privacy: A Simple and Practical Recipe](https://mlsanity.com/paper/2210.14348/) (2022)
- [Privacy Preserving Machine Learning: Threats and Solutions](https://mlsanity.com/paper/1804.11238/) (2018)
- [Learning Semantic Textual Similarity from Conversations](https://mlsanity.com/paper/1804.07754/) (2018)
- [Pre-trained Language Model Representations for Language Generation](https://mlsanity.com/paper/1903.09722/) (2019)
- [To Be Forgotten or To Be Fair: Unveiling Fairness Implications of Machine Unlearning Methods](https://mlsanity.com/paper/2302.03350/) (2023)

## Built on by (most cited first, 30 of 60)

- [TOFU: A Task of Fictitious Unlearning for LLMs](https://mlsanity.com/paper/2401.06121/) (2024)
- [Who's Harry Potter? Approximate Unlearning in LLMs](https://mlsanity.com/paper/2310.02238/) (2023)
- [Rethinking Machine Unlearning for Large Language Models](https://mlsanity.com/paper/2402.08787/) (2024)
- [Can Sensitive Information Be Deleted From LLMs? Objectives for Defending Against Extraction Attacks](https://mlsanity.com/paper/2309.17410/) (2023)
- [Guardrail Baselines for Unlearning in LLMs](https://mlsanity.com/paper/2403.03329/) (2024)
- [SILO Language Models: Isolating Legal Risk In a Nonparametric Datastore](https://mlsanity.com/paper/2308.04430/) (2023)
- [Position: LLM Unlearning Benchmarks are Weak Measures of Progress](https://mlsanity.com/paper/2410.02879/) (2024)
- [Rethinking LLM Unlearning Objectives: A Gradient Perspective and Go Beyond](https://mlsanity.com/paper/2502.19301/) (2025)
- [A Closer Look at Machine Unlearning for Large Language Models](https://mlsanity.com/paper/2410.08109/) (2024)
- [On Large Language Model Continual Unlearning](https://mlsanity.com/paper/2407.10223/) (2024)
- [Knowledge Unlearning for LLMs: Tasks, Methods, and Challenges](https://mlsanity.com/paper/2311.15766/) (2023)
- [Offset Unlearning for Large Language Models](https://mlsanity.com/paper/2404.11045/) (2024)
- [Advances and Open Challenges in Federated Foundation Models](https://mlsanity.com/paper/2404.15381/) (2024)
- [Benchmarking Vision Language Model Unlearning via Fictitious Facial Identity Dataset](https://mlsanity.com/paper/2411.03554/) (2024)
- [Mask-based Membership Inference Attacks for Retrieval-Augmented Generation](https://mlsanity.com/paper/2410.20142/) (2024)
- [Generative AI in EU Law: Liability, Privacy, Intellectual Property, and Cybersecurity](https://mlsanity.com/paper/2401.07348/) (2024)
- [Extracting Unlearned Information from LLMs with Activation Steering](https://mlsanity.com/paper/2411.02631/) (2024)
- [Unbridled Icarus: A Survey of the Potential Perils of Image Inputs in Multimodal Large Language Model Security](https://mlsanity.com/paper/2404.05264/) (2024)
- [The Frontier of Data Erasure: Machine Unlearning for Large Language Models](https://mlsanity.com/paper/2403.15779/) (2024)
- [Large Scale Knowledge Washing](https://mlsanity.com/paper/2405.16720/) (2024)
- [UnStar: Unlearning with Self-Taught Anti-Sample Reasoning for LLMs](https://mlsanity.com/paper/2410.17050/) (2024)
- [Machine Unlearning Doesn't Do What You Think: Lessons for Generative AI Policy and Research](https://mlsanity.com/paper/2412.06966/) (2024)
- [REX: Rapid Exploration and eXploitation for AI Agents](https://mlsanity.com/paper/2307.08962/) (2023)
- [GRU: Mitigating the Trade-off between Unlearning and Retention for LLMs](https://mlsanity.com/paper/2503.09117/) (2025)
- [Unlearning That Lasts: Utility-Preserving, Robust, and Almost Irreversible Forgetting in LLMs](https://mlsanity.com/paper/2509.02820/) (2025)
- [UPCORE: Utility-Preserving Coreset Selection for Balanced Unlearning](https://mlsanity.com/paper/2502.15082/) (2025)
- [Does Machine Unlearning Truly Remove Knowledge?](https://mlsanity.com/paper/2505.23270/) (2025)
- [Attention Smoothing Is All You Need For Unlearning](https://mlsanity.com/paper/2603.01285/) (2026)
- [MemPrivacy: Privacy-Preserving Personalized Memory Management for Edge-Cloud Agents](https://mlsanity.com/paper/2605.09530/) (2026)
- [Recent Advances in Attack and Defense Approaches of Large Language Models](https://mlsanity.com/paper/2409.03274/) (2024)

## Abstract (arXiv)

The Right to be Forgotten (RTBF) was first established as the result of the ruling of Google Spain SL, Google Inc. v AEPD, Mario Costeja Gonz'alez, and was later included as the Right to Erasure under the General Data Protection Regulation (GDPR) of European Union to allow individuals the right to request personal data be deleted by organizations. Specifically for search engines, individuals can send requests to organizations to exclude their information from the query results. It was a significant emergent right as the result of the evolution of technology. With the recent development of Large Language Models (LLMs) and their use in chatbots, LLM-enabled software systems have become popular. But they are not excluded from the RTBF. Compared with the indexing approach used by search engines, LLMs store, and process information in a completely different way. This poses new challenges for compliance with the RTBF. In this paper, we explore these challenges and provide our insights on how to implement technical solutions for the RTBF, including the use of differential privacy, machine unlearning, model editing, and guardrails. With the rapid advancement of AI and the increasing need of regulating this powerful technology, learning from the case of RTBF can provide valuable lessons for technical practitioners, legal experts, organizations, and authorities.