mlsanityData to 14 Oct 2026

Eraser: Jailbreaking Defense in Large Language Models via Unlearning Harmful Knowledge

arXiv 2404.05880 ↗First posted 8 Apr 2024cs.CLcited by 51

On the map: LLM safety and security → Jailbreaking

Abstract (arXiv)

Jailbreaking attacks can enable Large Language Models (LLMs) to bypass the safeguard and generate harmful content. Existing jailbreaking defense methods have failed to address the fundamental issue that harmful knowledge resides within the model, leading to potential jailbreak risks for LLMs. In this paper, we propose a novel defense method called Eraser, which mainly includes three goals: unlearning harmful knowledge, retaining general knowledge, and maintaining safety alignment. The intuition is that if an LLM forgets the specific knowledge required to answer a harmful question, it will no longer have the ability to answer harmful questions. The training of Erase does not actually require the model's own harmful knowledge, and it can benefit from unlearning general answers related to harmful queries, which means it does not need assistance from the red team. The experimental results show that Eraser can significantly reduce the jailbreaking success rate for various attacks without compromising the general capabilities of the model. Our codes are available at https://github.com/ZeroNLP/Eraser.

Key facts. Cited by 51 papers in the map (data through 14 Oct 2026). Placed in Jailbreaking, one of 12 directions in LLM safety and security. Builds on 39 of its 45 references that are in the map. First posted on arXiv 8 Apr 2024.

Builds on

39 of 45 references.

Closest ancestors · 16

Same direction, newest first

  1. 2024Play Guessing Game with LLM: Indirect Jailbreak Attack with Implicit Clues14 Feb
  2. 2024How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMs12 Jan
  3. 2023Tree of Attacks: Jailbreaking Black-Box LLMs Automatically4 Dec
  4. 2023Defending Large Language Models Against Jailbreaking Attacks Through Goal Prioritization15 Nov
  5. 2023Self-Guard: Empower the LLM to Safeguard Itself24 Oct
  6. 2023Jailbreaking Black Box Large Language Models in Twenty Queries12 Oct
  7. 2023SmoothLLM: Defending Large Language Models Against Jailbreaking Attacks5 Oct
  8. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models3 Oct
  9. 2023Jailbreaker in Jail: Moving Target Defense for Large Language Models3 Oct
  10. 2023GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Prompts19 Sep
  11. 2023Defending Against Alignment-Breaking Attacks via Robustly Aligned LLM18 Sep
  12. 2023FuzzLLM: A Novel and Universal Fuzzing Framework for Proactively Discovering Jailbreak Vulnerabilities in Large Language Models11 Sep
4 more
  1. 2023Certifying LLM Safety against Adversarial Prompting6 Sep
  2. 2023"Do Anything Now": Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Models7 Aug
  3. 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul
  4. 2023Jailbreaking ChatGPT via Prompt Engineering: An Empirical Study23 May

Other references · 23

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  5. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  6. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  7. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
  8. 2023Qwen Technical Reportcited by 3,344
  9. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,183
  10. 2023PaLM 2 Technical Reportcited by 1,162
  11. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,154
  12. 2019Machine Unlearningcited by 1,061
11 more
  1. 2023Baichuan 2: Open Large-scale Language Modelscited by 670
  2. 2022Knowledge Unlearning for Mitigating Privacy Risks in Language Modelscited by 437
  3. 2023Who's Harry Potter? Approximate Unlearning in LLMscited by 381
  4. 2023Toxicity in ChatGPT: Analyzing Persona-assigned Language Modelscited by 360
  5. 2022A Holistic Approach to Undesired Content Detection in the Real Worldcited by 356
  6. 2023Large Language Model Unlearningcited by 302
  7. 2023LLM Self Defense: By Self Examination, LLMs Know They Are Being Trickedcited by 244
  8. 2023Unlearn What You Want to Forget: Efficient Unlearning for LLMscited by 215
  9. 2023A Survey of Safety and Trustworthiness of Large Language Models through the Lens of Verification and Validationcited by 93
  10. 2023Attack Prompt Generation for Red Teaming and Defending Large Language Modelscited by 76
  11. 2023Making Harmful Behaviors Unlearnable for Large Language Modelscited by 20

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 19; 2025: 18; 2026: 14.

51 papers.

Sort:

By area:

  1. 2024Latent Adversarial Training Improves Robustness to Persistent Harmful Behaviors in LLMscited by 173
  2. 2024Uncovering Safety Risks of Large Language Models through Concept Activation Vectorcited by 67
  3. 2024TrojanRAG: Retrieval-Augmented Generation Can Be Backdoor Driver in Large Language Modelscited by 67
  4. 2024SelfDefend: LLMs Can Defend Themselves against Jailbreaking in a Practical Mannercited by 63
  5. 2024A Closer Look at Machine Unlearning for Large Language Modelscited by 56
  6. 2024Bag of Tricks: Benchmarking of Jailbreak Attacks on LLMscited by 51
  7. 2024Machine Unlearning in Generative AI: A Surveycited by 49
  8. 2024JAILJUDGE: A Comprehensive Jailbreak Judge Benchmark with Multi-Agent Enhanced Explanation Evaluation Frameworkcited by 47
  9. 2024Jailbreaking and Mitigation of Vulnerabilities in Large Language Modelscited by 34
  10. 2025Reasoning-to-Defend: Safety-Aware Reasoning Can Defend Large Language Models from Jailbreakingcited by 34
  11. 2024JailbreakEval: An Integrated Toolkit for Evaluating Jailbreak Attempts Against Large Language Modelscited by 31
  12. 2024From Theft to Bomb-Making: The Ripple Effect of Unlearning in Defending Against Jailbreak Attackscited by 29
38 more
  1. 2024To Forget or Not? Towards Practical Knowledge Unlearning for Large Language Modelscited by 26
  2. 2024Dissecting Fine-Tuning Unlearning in Large Language Modelscited by 26
  3. 2024Unified Parameter-Efficient Unlearning for LLMscited by 25
  4. 2025SAFEPATH: Preventing Harmful Reasoning in Chain-of-Thought via Early Alignmentcited by 24
  5. 2025ReLearn: Unlearning via Learning for Large Language Modelscited by 22
  6. 2025SoK: Evaluating Jailbreak Guardrails for Large Language Modelscited by 21
  7. 2025X-Boundary: Establishing Exact Safety Boundary to Shield LLMs from Multi-Turn Jailbreaks without Compromising Usabilitycited by 20
  8. 2024HiddenGuard: Fine-Grained Safe Generation with Specialized Representation Routercited by 16
  9. 2024Unveiling Entity-Level Unlearning for Large Language Models: A Comprehensive Analysiscited by 14
  10. 2025Safety Alignment via Constrained Knowledge Unlearningcited by 14
  11. 2025Self-Destructive Language Modelcited by 10
  12. 2025DRAGON: Guard LLM Unlearning in Context via Negative Detection and Reasoningcited by 7
  13. 2026SoK: Privacy Risks and Mitigations in Retrieval-Augmented Generation Systemscited by 7
  14. 2026Attention Smoothing Is All You Need For Unlearningcited by 7
  15. 2024Recent Advances in Attack and Defense Approaches of Large Language Modelscited by 6
  16. 2025Wisdom is Knowing What not to Say: Hallucination-Free LLMs Unlearning via Attention Shiftingcited by 6
  17. 2025SEPS: A Separability Measure for Robust Unlearning in LLMscited by 5
  18. 2025UniErase: Towards Balanced and Precise Unlearning in Language Modelscited by 5
  19. 2025Generative Model Unlearning: A Survey through Target Events, Unlearning Operators, and Evaluation Protocolscited by 4
  20. 2025SafeLLM: Unlearning Harmful Outputs from Large Language Models against Jailbreak Attackscited by 4
  21. 2025A Survey on Unlearning in Large Language Modelscited by 4
  22. 2026Jailbreaking Leaves a Trace: Understanding and Detecting Jailbreak Attacks from Internal Representations of Large Language Modelscited by 4
  23. 2026TrajGuard: Streaming Hidden-state Trajectory Detection for Decoding-time Jailbreak Defensecited by 4
  24. 2024Preventing Jailbreak Prompts as Malicious Tools for Cybercriminals: A Cyber Defense Perspectivecited by 3
  25. 2025STShield: Single-Token Sentinel for Real-Time Jailbreak Detection in Large Language Modelscited by 3
  26. 2024Wait, but Tylenol is Acetaminophen... Investigating and Improving Language Models' Ability to Resist Requests for Misinformationcited by 2
  27. 2025CoCoTen: Detecting Adversarial Inputs to Large Language Models through Latent Space Features of Contextual Co-occurrence Tensorscited by 2
  28. 2026KUDA: Knowledge Unlearning by Deviating Representation for Large Language Modelscited by 2
  29. 2025Automating Evaluation of Diffusion Model Unlearning with (Vision-) Language Model World Knowledgecited by 1
  30. 2026Isolation as a First-Class Principle for LLM-Agent System Safety: Concepts, Taxonomy, Challenges and Future Directionscited by 1
  31. 2025SoK: a Comprehensive Causality Analysis Framework for Large Language Model Securitycited by 0
  32. 2026JPU: Bridging Jailbreak Defense and Unlearning via On-Policy Path Rectificationcited by 0
  33. 2026Shadow Unlearning: A Neuro-Semantic Approach to Fidelity-Preserving Faceless Forgetting in LLMscited by 0
  34. 2026The Persistent Vulnerability of Aligned AI Systemscited by 0
  35. 2026Exclusive Unlearningcited by 0
  36. 2026Jailbreaking Frontier Foundation Models Through Intention Deceptioncited by 0
  37. 2026Safety Anchor: Defending Harmful Fine-tuning via Geometric Bottleneckscited by 0
  38. 2026Fast Unlearning at Scale via Margin Self-Correctioncited by 0

50 most cited of 51 All 51 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Knowledge-to-Jailbreak: Investigating Knowledge-driven Jailbreaking Attacks for Large Language Models
  2. 2024Jailbreak Attacks and Defenses Against Large Language Models: A Survey
  3. 2024A Comprehensive Study of Jailbreak Attack versus Defense for Large Language Models
  4. 2025Safe2Harm: Semantic Isomorphism Attacks for Jailbreaking Large Language Models
  5. 2025Unraveling LLM Jailbreaks Through Safety Knowledge Neurons
  6. 2025Harmful Prompt Laundering: Jailbreaking LLMs with Abductive Styles and Symbolic Encoding
  7. 2024Harnessing Task Overload for Scalable Jailbreak Attacks on Large Language Models
  8. 2025Rewrite to Jailbreak: Discover Learnable and Transferable Implicit Harmfulness Instruction