# The Erasure Illusion: Stress-Testing the Generalization of LLM Forgetting Evaluation

arXiv: https://arxiv.org/abs/2512.19025 · HTML: https://mlsanity.com/paper/2512.19025/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Machine unlearning, one of 12 directions in LLM safety and security.
- Builds on 33 of its 44 references that are in the map.
- First posted on arXiv 22 Dec 2025.

## Placed in

- LLM safety and security / [Machine unlearning](https://mlsanity.com/safety/machine-unlearning/)

## Builds on (most cited first, 30 of 33)

- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Pointer Sentinel Mixture Models](https://mlsanity.com/paper/1609.07843/) (2016)
- [Extracting Training Data from Large Language Models](https://mlsanity.com/paper/2012.07805/) (2020)
- [Editing Models with Task Arithmetic](https://mlsanity.com/paper/2212.04089/) (2022)
- [BERTopic: Neural topic modeling with a class-based TF-IDF procedure](https://mlsanity.com/paper/2203.05794/) (2022)
- [An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks](https://mlsanity.com/paper/1312.6211/) (2013)
- [Knowledge Unlearning for Mitigating Privacy Risks in Language Models](https://mlsanity.com/paper/2210.01504/) (2022)
- [Nuanced Metrics for Measuring Unintended Bias with Real Data for Text Classification](https://mlsanity.com/paper/1903.04561/) (2019)
- [Detecting Pretraining Data from Large Language Models](https://mlsanity.com/paper/2310.16789/) (2023)
- [Large Language Model Unlearning](https://mlsanity.com/paper/2310.10683/) (2023)
- [Rethinking Machine Unlearning for Large Language Models](https://mlsanity.com/paper/2402.08787/) (2024)
- [Learning to Ask: Neural Question Generation for Reading Comprehension](https://mlsanity.com/paper/1705.00106/) (2017)
- [Adaptive Machine Unlearning](https://mlsanity.com/paper/2106.04378/) (2021)
- [Can Sensitive Information Be Deleted From LLMs? Objectives for Defending Against Extraction Attacks](https://mlsanity.com/paper/2309.17410/) (2023)
- [An Adversarial Perspective on Machine Unlearning for AI Safety](https://mlsanity.com/paper/2409.18025/) (2024)
- [Copyright Violations and Large Language Models](https://mlsanity.com/paper/2310.13771/) (2023)
- [SOUL: Unlocking the Power of Second-Order Optimization for LLM Unlearning](https://mlsanity.com/paper/2404.18239/) (2024)
- [Rethinking LLM Memorization through the Lens of Adversarial Compression](https://mlsanity.com/paper/2404.15146/) (2024)
- [Reversing the Forget-Retain Objectives: An Efficient LLM Unlearning Framework from Logit Difference](https://mlsanity.com/paper/2406.08607/) (2024)
- [Inexact Unlearning Needs More Careful Evaluations to Avoid a False Sense of Privacy](https://mlsanity.com/paper/2403.01218/) (2024)
- [LLM Unlearning via Loss Adjustment with Only Forget Data](https://mlsanity.com/paper/2410.11143/) (2024)
- [Position: LLM Unlearning Benchmarks are Weak Measures of Progress](https://mlsanity.com/paper/2410.02879/) (2024)
- [Rethinking LLM Unlearning Objectives: A Gradient Perspective and Go Beyond](https://mlsanity.com/paper/2502.19301/) (2025)
- [Unveiling the Implicit Toxicity in Large Language Models](https://mlsanity.com/paper/2311.17391/) (2023)
- [Machine Unlearning in Generative AI: A Survey](https://mlsanity.com/paper/2407.20516/) (2024)
- [Unified Gradient-Based Machine Unlearning with Remain Geometry Enhancement](https://mlsanity.com/paper/2409.19732/) (2024)
- [To Each (Textual Sequence) Its Own: Improving Memorized-Data Unlearning in Large Language Models](https://mlsanity.com/paper/2405.03097/) (2024)
- [On Effects of Steering Latent Representation for Large Language Model Unlearning](https://mlsanity.com/paper/2408.06223/) (2024)
- [Towards Effective Evaluations and Comparisons for LLM Unlearning Methods](https://mlsanity.com/paper/2406.09179/) (2024)
- [Towards Scalable Exact Machine Unlearning Using Parameter-Efficient Fine-Tuning](https://mlsanity.com/paper/2406.16257/) (2024)

## Built on by (most cited first, 5 of 5)

- [Position: The Term "Machine Unlearning" Is Overused in LLMs](https://mlsanity.com/paper/2606.27379/) (2026)
- [Layer-Targeted Multilingual Knowledge Erasure in Large Language Models](https://mlsanity.com/paper/2602.22562/) (2026)
- [Probe-Geometry Alignment: Erasing the Cross-Sequence Memorization Signature Below Chance](https://mlsanity.com/paper/2605.01699/) (2026)
- [Probing Stylistic Appropriation using Large Language Models: An Evaluation Framework for Copyright Infringement under EU Law](https://mlsanity.com/paper/2606.31250/) (2026)
- [Can Scientific Claims Be Removed from Large Language Models? A Systematic Evaluation of Claim-Level Unlearning](https://mlsanity.com/paper/2608.20960/) (2026)

## Abstract (arXiv)

Machine unlearning aims to remove specific data influences from trained models, a capability essential for adhering to copyright laws and ensuring AI safety. Current unlearning metrics typically measure success by monitoring the model's performance degradation on the specific unlearning dataset (Du). We argue that for Large Language Models (LLMs), this evaluation paradigm is insufficient and potentially misleading. Many real-world uses of unlearning--motivated by copyright or safety--implicitly target not only verbatim content in Du, but also behaviors influenced by the broader generalizations the model derived from it. We demonstrate that LLMs can pass standard unlearning evaluation and appear to have "forgotten" the target knowledge, while simultaneously retaining strong capabilities on content that is semantically adjacent to Du. This phenomenon indicates that erasing exact sentences does not necessarily equate to removing the underlying knowledge. To address this gap, we propose Proximal Surrogate Generation (PSG), an automated stress-testing framework that generates a surrogate dataset, tildeDu. This surrogate set is constructed to be semantically derived from Du yet sufficiently distinct in embedding space. By comparing unlearning metric scores between Du and tildeDu, we can stress-test the reliability of the metric itself. Our extensive evaluation across three LLM families (Llama-3-8B, Qwen2.5-7B, and Zephyr-7B-β), three distinct datasets, and seven standard metrics reveals widespread inconsistencies. We find that current metrics frequently overestimate unlearning success, failing to detect retained knowledge exposed by our stress-test datasets.