# Towards Robust Knowledge Unlearning: An Adversarial Framework for Assessing and Improving Unlearning Robustness in Large Language Models

arXiv: https://arxiv.org/abs/2408.10682 · HTML: https://mlsanity.com/paper/2408.10682/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Placed in Machine unlearning, one of 12 directions in LLM safety and security.
- Builds on 19 of its 28 references that are in the map.
- First posted on arXiv 20 Aug 2024.

## Placed in

- LLM safety and security / [Machine unlearning](https://mlsanity.com/safety/machine-unlearning/)

## Builds on (most cited first, 19 of 19)

- [Universal and Transferable Adversarial Attacks on Aligned Language Models](https://mlsanity.com/paper/2307.15043/) (2023)
- [TOFU: A Task of Fictitious Unlearning for LLMs](https://mlsanity.com/paper/2401.06121/) (2024)
- [Negative Preference Optimization: From Catastrophic Collapse to Effective Unlearning](https://mlsanity.com/paper/2404.05868/) (2024)
- [Catastrophic Jailbreak of Open-source LLMs via Exploiting Generation](https://mlsanity.com/paper/2310.06987/) (2023)
- [Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrations](https://mlsanity.com/paper/2310.06387/) (2023)
- [SalUn: Empowering Machine Unlearning via Gradient-based Weight Saliency in Both Image Classification and Generation](https://mlsanity.com/paper/2310.12508/) (2023)
- [Who's Harry Potter? Approximate Unlearning in LLMs](https://mlsanity.com/paper/2310.02238/) (2023)
- [MUSE: Machine Unlearning Six-Way Evaluation for Language Models](https://mlsanity.com/paper/2407.06460/) (2024)
- [Unlearn What You Want to Forget: Efficient Unlearning for LLMs](https://mlsanity.com/paper/2310.20150/) (2023)
- [Model Sparsity Can Simplify Machine Unlearning](https://mlsanity.com/paper/2304.04934/) (2023)
- [Automatically Auditing Large Language Models via Discrete Optimization](https://mlsanity.com/paper/2303.04381/) (2023)
- [Can Sensitive Information Be Deleted From LLMs? Objectives for Defending Against Extraction Attacks](https://mlsanity.com/paper/2309.17410/) (2023)
- [AdvPrompter: Fast Adaptive Adversarial Prompting for LLMs](https://mlsanity.com/paper/2404.16873/) (2024)
- [Efficient Adversarial Training in LLMs with Continuous Attacks](https://mlsanity.com/paper/2405.15589/) (2024)
- [Defending Against Unforeseen Failure Modes with Latent Adversarial Training](https://mlsanity.com/paper/2403.05030/) (2024)
- [Unlearning or Obfuscating? Jogging the Memory of Unlearned LLMs via Benign Relearning](https://mlsanity.com/paper/2406.13356/) (2024)
- [UnUnlearning: Unlearning is not sufficient for content regulation in advanced generative AI](https://mlsanity.com/paper/2407.00106/) (2024)
- [Machine Unlearning in Generative AI: A Survey](https://mlsanity.com/paper/2407.20516/) (2024)
- [Evaluating Copyright Takedown Methods for Language Models](https://mlsanity.com/paper/2406.18664/) (2024)

## Built on by (most cited first, 27 of 27)

- [Position: LLM Unlearning Benchmarks are Weak Measures of Progress](https://mlsanity.com/paper/2410.02879/) (2024)
- [Meta-Unlearning on Diffusion Models: Preventing Relearning Unlearned Concepts](https://mlsanity.com/paper/2410.12777/) (2024)
- [A Survey on Privacy Risks and Protection in Large Language Models](https://mlsanity.com/paper/2505.01976/) (2025)
- [GUARD: Generation-time LLM Unlearning via Adaptive Restriction and Detection](https://mlsanity.com/paper/2505.13312/) (2025)
- [RULE: Reinforcement UnLEarning Achieves Forget-Retain Pareto Optimality](https://mlsanity.com/paper/2506.07171/) (2025)
- [SoK: Machine Unlearning for Large Language Models](https://mlsanity.com/paper/2506.09227/) (2025)
- [Omni-Reward: Towards Generalist Omni-Modal Reward Modeling with Free-Form Preferences](https://mlsanity.com/paper/2510.23451/) (2025)
- [Improving LLM Unlearning Robustness via Random Perturbations](https://mlsanity.com/paper/2501.19202/) (2025)
- [A Guardrail for Safety Preservation: When Safety-Sensitive Subspace Meets Harmful-Resistant Null-Space](https://mlsanity.com/paper/2510.14301/) (2025)
- [Prompt Attacks Reveal Superficial Knowledge Removal in Unlearning Methods](https://mlsanity.com/paper/2506.10236/) (2025)
- [How Secure is Forgetting? Linking Machine Unlearning to Machine Learning Attacks](https://mlsanity.com/paper/2503.20257/) (2025)
- [Large Language Model Unlearning for Source Code](https://mlsanity.com/paper/2506.17125/) (2025)
- [SafeLLM: Unlearning Harmful Outputs from Large Language Models against Jailbreak Attacks](https://mlsanity.com/paper/2508.15182/) (2025)
- [Downgrade to Upgrade: Optimizer Simplification Enhances Robustness in LLM Unlearning](https://mlsanity.com/paper/2510.00761/) (2025)
- [REBEL: Hidden Knowledge Recovery via Evolutionary-Based Evaluation Loop](https://mlsanity.com/paper/2602.06248/) (2026)
- [Dual-Space Smoothness for Robust and Balanced LLM Unlearning](https://mlsanity.com/paper/2509.23362/) (2025)
- [Position: Capability Control Should be a Separate Goal From Alignment](https://mlsanity.com/paper/2602.05164/) (2026)
- [RUB: Evaluating Residual Knowledge in Unlearned Models](https://mlsanity.com/paper/2504.14798/) (2025)
- [Learning-Time Encoding Shapes Unlearning in LLMs](https://mlsanity.com/paper/2506.15076/) (2025)
- [Game-Theoretic Machine Unlearning: Mitigating Extra Privacy Leakage](https://mlsanity.com/paper/2411.03914/) (2024)
- [Consistency-Aware Editing for Entity-level Unlearning in Language Models](https://mlsanity.com/paper/2601.08840/) (2025)
- [EvoMU: Evolutionary Machine Unlearning](https://mlsanity.com/paper/2602.02139/) (2026)
- [Unlearning Offline Stochastic Multi-Armed Bandits](https://mlsanity.com/paper/2605.00638/) (2026)
- [POPS: Recovering Unlearned Multi-Modality Knowledge in MLLMs with Prompt-Optimized Parameter Shaking](https://mlsanity.com/paper/2607.06649/) (2026)
- [Test-Time Unlearning via Sparse Autoencoder](https://mlsanity.com/paper/2609.16229/) (2026)
- [Damnatio Memoriae: Adversarially and Selectively Forgetting Identities in the Embedding Space of Face Recognition Models](https://mlsanity.com/paper/2609.27115/) (2026)
- [Do LLMs Really Forget? Hidden-State Leakage in Model Unlearning and How to Fix it](https://mlsanity.com/paper/2609.36612/) (2026)

## Abstract (arXiv)

LLM have achieved success in many fields but still troubled by problematic content in the training corpora. LLM unlearning aims at reducing their influence and avoid undesirable behaviours. However, existing unlearning methods remain vulnerable to adversarial queries and the unlearned knowledge resurfaces after the manually designed attack queries. As part of a red-team effort to proactively assess the vulnerabilities of unlearned models, we design Dynamic Unlearning Attack (DUA), a dynamic and automated framework to attack these models and evaluate their robustness. It optimizes adversarial suffixes to reintroduce the unlearned knowledge in various scenarios. We find that unlearned knowledge can be recovered in 55.2% of the questions, even without revealing the unlearned model's parameters. In response to this vulnerability, we propose Latent Adversarial Unlearning (LAU), a universal framework that effectively enhances the robustness of the unlearned process. It formulates the unlearning process as a min-max optimization problem and resolves it through two stages: an attack stage, where perturbation vectors are trained and added to the latent space of LLMs to recover the unlearned knowledge, and a defense stage, where previously trained perturbation vectors are used to enhance unlearned model's robustness. With our LAU framework, we obtain two robust unlearning methods, AdvGA and AdvNPO. We conduct extensive experiments across multiple unlearning benchmarks and various models, and demonstrate that they improve the unlearning effectiveness by over 53.5%, cause only less than a 11.6% reduction in neighboring knowledge, and have almost no impact on the model's general capabilities.