# Unlearning vs. Obfuscation: Are We Truly Removing Knowledge?

arXiv: https://arxiv.org/abs/2505.02884 · HTML: https://mlsanity.com/paper/2505.02884/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Placed in Machine unlearning, one of 12 directions in LLM safety and security.
- Builds on 10 of its 19 references that are in the map.
- First posted on arXiv 5 May 2025.

## Placed in

- LLM safety and security / [Machine unlearning](https://mlsanity.com/safety/machine-unlearning/)

## Builds on (most cited first, 10 of 10)

- [TOFU: A Task of Fictitious Unlearning for LLMs](https://mlsanity.com/paper/2401.06121/) (2024)
- [Negative Preference Optimization: From Catastrophic Collapse to Effective Unlearning](https://mlsanity.com/paper/2404.05868/) (2024)
- [Memory-Based Model Editing at Scale](https://mlsanity.com/paper/2206.06520/) (2022)
- [Knowledge Unlearning for Mitigating Privacy Risks in Language Models](https://mlsanity.com/paper/2210.01504/) (2022)
- [Who's Harry Potter? Approximate Unlearning in LLMs](https://mlsanity.com/paper/2310.02238/) (2023)
- [Unlearn What You Want to Forget: Efficient Unlearning for LLMs](https://mlsanity.com/paper/2310.20150/) (2023)
- [KGA: A General Machine Unlearning Framework Based on Knowledge Gap Alignment](https://mlsanity.com/paper/2305.06535/) (2023)
- [LLM Unlearning via Loss Adjustment with Only Forget Data](https://mlsanity.com/paper/2410.11143/) (2024)
- [Revisiting Who's Harry Potter: Towards Targeted Unlearning from a Causal Intervention Perspective](https://mlsanity.com/paper/2407.16997/) (2024)
- [UnStar: Unlearning with Self-Taught Anti-Sample Reasoning for LLMs](https://mlsanity.com/paper/2410.17050/) (2024)

## Built on by (most cited first, 11 of 11)

- [LLM-based Agents Suffer from Hallucinations: A Survey of Taxonomy, Methods, and Directions](https://mlsanity.com/paper/2509.18970/) (2025)
- [Watch the Weights: Unsupervised monitoring and control of fine-tuned LLMs](https://mlsanity.com/paper/2508.00161/) (2025)
- [Do Bias Benchmarks Generalise? Evidence from Voice-based Evaluation of Gender Bias in SpeechLLMs](https://mlsanity.com/paper/2510.01254/) (2025)
- [Reference-Specific Unlearning Metrics Can Hide the Truth: A Reality Check](https://mlsanity.com/paper/2510.12981/) (2025)
- [LACUNA: A Testbed for Evaluating Localization Precision for LLM Unlearning](https://mlsanity.com/paper/2607.02513/) (2026)
- [Representation-Aware Unlearning via Activation Signatures: From Suppression to Entity-Signature Erasure](https://mlsanity.com/paper/2601.10566/) (2026)
- [Privacy and Safety Experiences and Concerns of U.S. Women Using Generative AI for Seeking Sexual and Reproductive Health Information](https://mlsanity.com/paper/2603.16918/) (2026)
- [Can Large Language Models Reinvent Foundational Algorithms?](https://mlsanity.com/paper/2604.05716/) (2026)
- [LLM Unlearning for Cyber Defense: A Survey on Methods, Challenges, and Emerging Threats](https://mlsanity.com/paper/2607.16227/) (2026)
- [DECAF: De-Clustering for Adaptive Representational Unlearning](https://mlsanity.com/paper/2607.23934/) (2026)
- [Cascade: Hierarchical Recoverability Control for Large Language Model Unlearning](https://mlsanity.com/paper/2609.16890/) (2026)

## Abstract (arXiv)

Unlearning has emerged as a critical capability for large language models (LLMs) to support data privacy, regulatory compliance, and ethical AI deployment. Recent techniques often rely on obfuscation by injecting incorrect or irrelevant information to suppress knowledge. Such methods effectively constitute knowledge addition rather than true removal, often leaving models vulnerable to probing. In this paper, we formally distinguish unlearning from obfuscation and introduce a probing-based evaluation framework to assess whether existing approaches genuinely remove targeted information. Moreover, we propose DF-MCQ, a novel unlearning method that flattens the model predictive distribution over automatically generated multiple-choice questions using KL-divergence, effectively removing knowledge about target individuals and triggering appropriate refusal behaviour. Experimental results demonstrate that DF-MCQ achieves unlearning with over 90% refusal rate and a random choice-level uncertainty that is much higher than obfuscation on probing questions.