# Does Unlearning Truly Unlearn? A Black Box Evaluation of LLM Unlearning Methods

arXiv: https://arxiv.org/abs/2411.12103 · HTML: https://mlsanity.com/paper/2411.12103/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Placed in Machine unlearning, one of 12 directions in LLM safety and security.
- Builds on 9 of its 14 references that are in the map.
- First posted on arXiv 18 Nov 2024.

## Placed in

- LLM safety and security / [Machine unlearning](https://mlsanity.com/safety/machine-unlearning/)

## Builds on (most cited first, 9 of 9)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Ethical and social risks of harm from Language Models](https://mlsanity.com/paper/2112.04359/) (2021)
- [The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearning](https://mlsanity.com/paper/2403.03218/) (2024)
- [Zephyr: Direct Distillation of LM Alignment](https://mlsanity.com/paper/2310.16944/) (2023)
- [Negative Preference Optimization: From Catastrophic Collapse to Effective Unlearning](https://mlsanity.com/paper/2404.05868/) (2024)
- [An Adversarial Perspective on Machine Unlearning for AI Safety](https://mlsanity.com/paper/2409.18025/) (2024)
- [Preference Tuning For Toxicity Mitigation Generalizes Across Languages](https://mlsanity.com/paper/2406.16235/) (2024)

## Built on by (most cited first, 26 of 26)

- [OpenUnlearning: Accelerating LLM Unlearning via Unified Benchmarking of Methods and Metrics](https://mlsanity.com/paper/2506.12618/) (2025)
- [SafeEraser: Enhancing Safety in Multimodal Large Language Models through Multimodal Machine Unlearning](https://mlsanity.com/paper/2502.12520/) (2025)
- [GUARD: Generation-time LLM Unlearning via Adaptive Restriction and Detection](https://mlsanity.com/paper/2505.13312/) (2025)
- [A General Framework to Enhance Fine-tuning-based LLM Unlearning](https://mlsanity.com/paper/2502.17823/) (2025)
- [Precise In-Parameter Concept Erasure in Large Language Models](https://mlsanity.com/paper/2505.22586/) (2025)
- [Improving LLM Unlearning Robustness via Random Perturbations](https://mlsanity.com/paper/2501.19202/) (2025)
- [Keeping an Eye on LLM Unlearning: The Hidden Risk and Remedy](https://mlsanity.com/paper/2506.00359/) (2025)
- [Does Machine Unlearning Truly Remove Knowledge?](https://mlsanity.com/paper/2505.23270/) (2025)
- [SEPS: A Separability Measure for Robust Unlearning in LLMs](https://mlsanity.com/paper/2505.14832/) (2025)
- [Prompt Attacks Reveal Superficial Knowledge Removal in Unlearning Methods](https://mlsanity.com/paper/2506.10236/) (2025)
- [DUSK: Do Not Unlearn Shared Knowledge](https://mlsanity.com/paper/2505.15209/) (2025)
- [An Example Safety Case for Safeguards Against Misuse](https://mlsanity.com/paper/2505.18003/) (2025)
- [SUA: Stealthy Multimodal Large Language Model Unlearning Attack](https://mlsanity.com/paper/2506.17265/) (2025)
- [A Survey on Unlearning in Large Language Models](https://mlsanity.com/paper/2510.25117/) (2025)
- [Towards Benchmarking Privacy Vulnerabilities in Selective Forgetting with Large Language Models](https://mlsanity.com/paper/2512.18035/) (2025)
- [Module-Aware Parameter-Efficient Machine Unlearning on Transformers](https://mlsanity.com/paper/2508.17233/) (2025)
- [Soft Token Attacks Cannot Reliably Audit Unlearning in Large Language Models](https://mlsanity.com/paper/2502.15836/) (2025)
- [Leveraging Machine Unlearning for Cost-Efficient Preference Alignment](https://mlsanity.com/paper/2504.06659/) (2025)
- [Beyond Forgetting: Representation Misdirection Elicits Controllable Side Behaviors and Capabilities](https://mlsanity.com/paper/2601.21702/) (2026)
- [A Comprehensive Evaluation of LLM Unlearning Robustness under Multi-Turn Interaction](https://mlsanity.com/paper/2603.00823/) (2026)
- [Suppression Is Not Forgetting: Residual Recoverability in Visual Concept Unlearning for VLMs](https://mlsanity.com/paper/2604.03114/) (2026)
- [From Anchors to Supervision: Memory-Graph Guided Corpus-Free Unlearning for Large Language Models](https://mlsanity.com/paper/2604.13777/) (2026)
- [Unlearners Can Lie: Evaluating and Improving Honesty in LLM Unlearning](https://mlsanity.com/paper/2605.08765/) (2026)
- [Unlearning as Distribution Restoration: A Controlled Counterfactual Study, a Validated Selective Screen, and the Limits of Oracle-Free Certification](https://mlsanity.com/paper/2607.19442/) (2026)
- [Stochastic Meta-Unlearning: Bridging Language Backbone and Multimodal Unlearning](https://mlsanity.com/paper/2607.18615/) (2026)
- [How Context Attribution Handles What the Model Already Knows](https://mlsanity.com/paper/2607.23804/) (2026)

## Abstract (arXiv)

Large language model unlearning aims to remove harmful information that LLMs have learnt to prevent their use for malicious purposes. LLMU and RMU have been proposed as two methods for LLM unlearning, achieving impressive results on unlearning benchmarks. We study in detail the impact of unlearning on LLM performance metrics using the WMDP dataset as well as a new biology dataset we create. We show that unlearning has a notable impact on general model capabilities, with the performance degradation being more significant in general for LLMU. We further test the robustness of the two methods and find that doing 5-shot prompting or rephrasing the question in simple ways can lead to an over ten-fold increase in accuracy on unlearning benchmarks. Finally, we show that training on unrelated data can almost completely recover pre-unlearning performance, demonstrating that these methods fail at truly unlearning. Our methodology serves as an evaluation framework for LLM unlearning methods. The code is available at: https://github.com/JaiDoshi/Knowledge-Erasure.