# Unlearning Isn't Deletion: Investigating Reversibility of Machine Unlearning in LLMs

arXiv: https://arxiv.org/abs/2505.16831 · HTML: https://mlsanity.com/paper/2505.16831/

## Key facts

- Cited by 42 papers in the map (data through 14 Oct 2026).
- Placed in Machine unlearning, one of 12 directions in LLM safety and security.
- Builds on 46 of its 58 references that are in the map.
- First posted on arXiv 22 May 2025.

## Placed in

- LLM safety and security / [Machine unlearning](https://mlsanity.com/safety/machine-unlearning/)

## Builds on (most cited first, 30 of 46)

- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Universal and Transferable Adversarial Attacks on Aligned Language Models](https://mlsanity.com/paper/2307.15043/) (2023)
- [CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge](https://mlsanity.com/paper/1811.00937/) (2018)
- [Similarity of Neural Network Representations Revisited](https://mlsanity.com/paper/1905.00414/) (2019)
- [Machine Unlearning](https://mlsanity.com/paper/1912.03817/) (2019)
- [Membership Inference Attacks From First Principles](https://mlsanity.com/paper/2112.03570/) (2021)
- [Yi: Open Foundation Models by 01.AI](https://mlsanity.com/paper/2403.04652/) (2024)
- [TOFU: A Task of Fictitious Unlearning for LLMs](https://mlsanity.com/paper/2401.06121/) (2024)
- [The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearning](https://mlsanity.com/paper/2403.03218/) (2024)
- [Negative Preference Optimization: From Catastrophic Collapse to Effective Unlearning](https://mlsanity.com/paper/2404.05868/) (2024)
- [Jailbreaking ChatGPT via Prompt Engineering: An Empirical Study](https://mlsanity.com/paper/2305.13860/) (2023)
- [Making AI Forget You: Data Deletion in Machine Learning](https://mlsanity.com/paper/1907.05012/) (2019)
- [Knowledge Unlearning for Mitigating Privacy Risks in Language Models](https://mlsanity.com/paper/2210.01504/) (2022)
- [Scalable Extraction of Training Data from (Production) Language Models](https://mlsanity.com/paper/2311.17035/) (2023)
- [Detecting Pretraining Data from Large Language Models](https://mlsanity.com/paper/2310.16789/) (2023)
- [Who's Harry Potter? Approximate Unlearning in LLMs](https://mlsanity.com/paper/2310.02238/) (2023)
- [MUSE: Machine Unlearning Six-Way Evaluation for Language Models](https://mlsanity.com/paper/2407.06460/) (2024)
- [In-Context Unlearning: Language Models as Few Shot Unlearners](https://mlsanity.com/paper/2310.07579/) (2023)
- [Language model compression with weighted low-rank factorization](https://mlsanity.com/paper/2207.00112/) (2022)
- [Do Membership Inference Attacks Work on Large Language Models?](https://mlsanity.com/paper/2402.07841/) (2024)
- [Eight Methods to Evaluate Robust Unlearning in LLMs](https://mlsanity.com/paper/2402.16835/) (2024)
- [Can Sensitive Information Be Deleted From LLMs? Objectives for Defending Against Extraction Attacks](https://mlsanity.com/paper/2309.17410/) (2023)
- [Machine Unlearning of Pre-trained Large Language Models](https://mlsanity.com/paper/2402.15159/) (2024)
- [Large Language Model Unlearning via Embedding-Corrupted Prompts](https://mlsanity.com/paper/2406.07933/) (2024)
- [Copyright Violations and Large Language Models](https://mlsanity.com/paper/2310.13771/) (2023)

## Built on by (most cited first, 30 of 42)

- [Does Math Reasoning Improve General LLM Capabilities? Understanding Transferability of LLM Reasoning](https://mlsanity.com/paper/2507.00432/) (2025)
- [Distillation Robustifies Unlearning](https://mlsanity.com/paper/2506.06278/) (2025)
- [SoK: Machine Unlearning for Large Language Models](https://mlsanity.com/paper/2506.09227/) (2025)
- [Beyond Scaling Law: A Data-Efficient Distillation Framework for Reasoning](https://mlsanity.com/paper/2508.09883/) (2025)
- [Model Immunization from a Condition Number Perspective](https://mlsanity.com/paper/2505.23760/) (2025)
- [Hair-Trigger Alignment: Black-Box Evaluation Cannot Guarantee Post-Update Alignment](https://mlsanity.com/paper/2601.22313/) (2026)
- [REBEL: Hidden Knowledge Recovery via Evolutionary-Based Evaluation Loop](https://mlsanity.com/paper/2602.06248/) (2026)
- [SineProject: Machine Unlearning for Stable Vision Language Alignment](https://mlsanity.com/paper/2511.18444/) (2025)
- [Can an AI Assistant Really Forget? Auditable Deletion from Addressable Memory](https://mlsanity.com/paper/2607.27539/) (2026)
- [Stored in Optimizer State, Valued by Later Training: A Causal Account of Subliminal Trait Transfer](https://mlsanity.com/paper/2608.20442/) (2026)
- [One-Point Contraction: Erasing Representational Separability toward Irreversible Deep Forgetting](https://mlsanity.com/paper/2507.07754/) (2025)
- [Improving Latent Reasoning in LLMs via Soft Concept Mixing](https://mlsanity.com/paper/2511.16885/) (2025)
- [From Domains to Instances: Dual-Granularity Data Synthesis for LLM Unlearning](https://mlsanity.com/paper/2601.04278/) (2026)
- [Mitigating Memorization Where It Happens](https://mlsanity.com/paper/2602.08901/) (2026)
- [Multilingual Unlearning in LLMs: Transfer, Dynamics, and Reversibility](https://mlsanity.com/paper/2606.03291/) (2026)
- [LACUNA: A Testbed for Evaluating Localization Precision for LLM Unlearning](https://mlsanity.com/paper/2607.02513/) (2026)
- [Unlearning at Scale: State-Exact Trace-Preserving Deletion in Billion-Parameter Language Models](https://mlsanity.com/paper/2508.12220/) (2025)
- [What the "Spotless" Mind Remembers: How Knowledge Entanglement Shapes What Leaks After Unlearning in LLMs](https://mlsanity.com/paper/2510.25732/) (2025)
- ["The Whole Is Greater Than the Sum of Its Parts": A Compatibility-Aware Multi-Teacher CoT Distillation Framework](https://mlsanity.com/paper/2601.13992/) (2026)
- [Beyond Forgetting: Representation Misdirection Elicits Controllable Side Behaviors and Capabilities](https://mlsanity.com/paper/2601.21702/) (2026)
- [Rethinking Benign Relearning: Syntax as the Hidden Driver of Unlearning Failures](https://mlsanity.com/paper/2602.03379/) (2026)
- [Copyright Detective: A Forensic System to Evidence LLMs Flickering Copyright Leakage Risks](https://mlsanity.com/paper/2602.05252/) (2026)
- [Anatomy of Unlearning: The Dual Impact of Fact Salience and Model Fine-Tuning](https://mlsanity.com/paper/2602.19612/) (2026)
- [Forgetting That Sticks: Quantization-Permanent Unlearning via Circuit Attribution](https://mlsanity.com/paper/2605.15138/) (2026)
- [Measuring the Depth of LLM Unlearning via Activation Patching](https://mlsanity.com/paper/2605.24614/) (2026)
- [GROM: Gradient-Free Rapid One-Shot Machine Unlearning](https://mlsanity.com/paper/2608.05783/) (2026)
- [Grokked Models are Better Unlearners](https://mlsanity.com/paper/2512.03437/) (2025)
- [GRIP: Algorithm-Agnostic Machine Unlearning for Mixture-of-Experts via Geometric Router Constraints](https://mlsanity.com/paper/2601.16905/) (2026)
- [FIT to Forget: Robust Continual Unlearning for Large Language Models](https://mlsanity.com/paper/2601.21682/) (2026)
- [textbfAGTAO: Robust and Stabilized LLM Unlearning via Adversarial Gating Training with Adaptive Orthogonality](https://mlsanity.com/paper/2602.01703/) (2026)

## Abstract (arXiv)

Unlearning in large language models (LLMs) aims to remove specified data, but its efficacy is typically assessed with task-level metrics like accuracy and perplexity. We show that these metrics can be misleading, as models can appear to forget while their original behavior is easily restored through minimal fine-tuning. This reversibility suggests that information is merely suppressed, not genuinely erased. To address this critical evaluation gap, we introduce a representation-level analysis framework. Our toolkit comprises PCA similarity and shift, centered kernel alignment (CKA), and Fisher information, complemented by a summary metric, the mean PCA distance, to measure representational drift. Applying this framework across multiple unlearning methods, data domains, and LLMs, we identify four distinct forgetting regimes based on their reversibility and catastrophicity. We compare recovery strategies and show that relearning efficiency relies on the data source. We also find that irreversible, non-catastrophic forgetting is exceptionally challenging. By probing unlearning limits, we identify a case of seemingly irreversible, targeted forgetting, offering insights for more robust erasure algorithms. Overall, our findings expose a gap in current evaluation and establish a representation-level foundation for trustworthy unlearning.