mlsanityData to 7 Oct 2026

Defending Large Language Models Against Jailbreak Attacks via Layer-specific Editing

arXiv 2405.18166 ↗First posted 28 May 2024cs.AIcited by 77

On the map: LLM safety and security → Jailbreaking

Abstract (arXiv)

Large language models (LLMs) are increasingly being adopted in a wide range of real-world applications. Despite their impressive performance, recent studies have shown that LLMs are vulnerable to deliberately crafted adversarial prompts even when aligned via Reinforcement Learning from Human Feedback or supervised fine-tuning. While existing defense methods focus on either detecting harmful prompts or reducing the likelihood of harmful responses through various means, defending LLMs against jailbreak attacks based on the inner mechanisms of LLMs remains largely unexplored. In this work, we investigate how LLMs response to harmful prompts and propose a novel defense method termed Layer-specific Editing (LED) to enhance the resilience of LLMs against jailbreak attacks. Through LED, we reveal that several critical textitsafety layers exist among the early layers of LLMs. We then show that realigning these safety layers (and some selected additional layers) with the decoded safe response from selected target layers can significantly improve the alignment of LLMs against jailbreak attacks. Extensive experiments across various LLMs (e.g., Llama2, Mistral) show the effectiveness of LED, which effectively defends against jailbreak attacks while maintaining performance on benign prompts. Our code is available at urlhttps://github.com/ledllm/ledllm.

Key facts. Cited by 77 papers in the map (data through 7 Oct 2026). Placed in Jailbreaking, one of 12 directions in LLM safety and security. Builds on 40 of its 55 references that are in the map. First posted on arXiv 28 May 2024.

Builds on

40 of 55 references.

Closest ancestors · 17

Same direction, newest first

  1. 2024EasyJailbreak: A Unified Framework for Jailbreaking Large Language Models18 Mar
  2. 2024Is the System Message Really Important to Jailbreaks in Large Language Models?20 Feb
  3. 2024Query-Based Adversarial Prompt Generation19 Feb
  4. 2024SafeDecoding: Defending against Jailbreak Attacks via Safety-Aware Decoding14 Feb
  5. 2024How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMs12 Jan
  6. 2023Causality Analysis for Evaluating the Security of Large Language Models13 Dec
  7. 2023DeepInception: Hypnotize Large Language Model to Be Jailbreaker6 Nov
  8. 2023Jailbreaking Black Box Large Language Models in Twenty Queries12 Oct
  9. 2023Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrations10 Oct
  10. 2023SmoothLLM: Defending Large Language Models Against Jailbreaking Attacks5 Oct
  11. 2023GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Prompts19 Sep
  12. 2023Defending Against Alignment-Breaking Attacks via Robustly Aligned LLM18 Sep
5 more
  1. 2023Detecting Language Model Attacks with Perplexity27 Aug
  2. 2023"Do Anything Now": Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Models7 Aug
  3. 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul
  4. 2023MasterKey: Automated Jailbreak Across Multiple Large Language Model Chatbots16 Jul
  5. 2023Jailbroken: How Does LLM Safety Training Fail?5 Jul

Other references · 23

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  5. 2023Mistral 7Bcited by 3,065
  6. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,307
  7. 2022Red Teaming Language Models with Language Modelscited by 983
  8. 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Modelscited by 608
  9. 2021Editing Factual Knowledge in Language Modelscited by 548
  10. 2022Improving alignment of dialogue agents via targeted human judgementscited by 527
  11. 2022Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Spacecited by 522
  12. 2024ShortGPT: Layers in Large Language Models are More Redundant Than You Expectcited by 369
11 more
  1. 2023The Unlocking Spell on Base LLMs: Rethinking Alignment via In-Context Learningcited by 251
  2. 2023LLM Self Defense: By Self Examination, LLMs Know They Are Being Trickedcited by 226
  3. 2020Modifying Memories in Transformer Modelscited by 222
  4. 2023RAIN: Your Language Models Can Align Themselves without Finetuningcited by 149
  5. 2023Future Lens: Anticipating Subsequent Tokens from a Single Hidden Statecited by 116
  6. 2024Not All Layers of LLMs Are Necessary During Inferencecited by 95
  7. 2023DEPN: Detecting and Editing Privacy Neurons in Pretrained Language Modelscited by 75
  8. 2024How Large Language Models Encode Context Knowledge? A Layer-Wise Probing Studycited by 64
  9. 2023Aligning Large Language Models with Human: A Surveycited by 31
  10. 2022Plug-and-Play Adaptation for Continuously-updated QAcited by 21
  11. 2023Forgetting before Learning: Utilizing Parametric Arithmetic for Knowledge Updating in Large Language Modelscited by 15

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 14; 2025: 36; 2026: 27.

77 papers.

Sort:

By area:

  1. 2025AgentSpec: Customizable Runtime Enforcement for Safe and Reliable LLM Agentscited by 196
  2. 2024Safety Layers in Aligned Large Language Models: The Key to LLM Securitycited by 126
  3. 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 110
  4. 2024On the Role of Attention Heads in Large Language Model Safetycited by 70
  5. 2024SelfDefend: LLMs Can Defend Themselves against Jailbreaking in a Practical Mannercited by 57
  6. 2024Multi-Turn Context Jailbreak Attack on Large Language Models From First Principlescited by 54
  7. 2024Bag of Tricks: Benchmarking of Jailbreak Attacks on LLMscited by 47
  8. 2024Operationalizing a Threat Model for Red-Teaming Large Language Models (LLMs)cited by 44
  9. 2025HiddenDetect: Detecting Jailbreak Attacks against Large Vision-Language Models via Monitoring Hidden Statescited by 29
  10. 2024Shaping the Safety Boundaries: Understanding and Defending Against Jailbreaks in Large Language Modelscited by 23
  11. 2025How Should We Enhance the Safety of Large Reasoning Models: An Empirical Studycited by 18
  12. 2025NeuroStrike: Neuron-Level Attacks on Aligned LLMscited by 18
38 more
  1. 2024Towards Inference-time Category-wise Safety Steering for Large Language Modelscited by 17
  2. 2025Understanding and Rectifying Safety Perception Distortion in VLMscited by 17
  3. 2025SoK: Evaluating Jailbreak Guardrails for Large Language Modelscited by 17
  4. 2025Position: Editing Large Language Models Poses Serious Safety Riskscited by 15
  5. 2025On Almost Surely Safe Alignment of Large Language Models at Inference-Timecited by 13
  6. 2025AISafetyLab: A Comprehensive Framework for AI Safety Evaluation and Improvementcited by 11
  7. 2025SoK: Trust-Authorization Mismatch in LLM Agent Interactionscited by 11
  8. 2025Interpretation Meets Safety: A Survey on Interpretation Methods and Tools for Improving LLM Safetycited by 10
  9. 2026LLM Safety From Within: Detecting Harmful Content with Internal Representationscited by 10
  10. 2025DELMAN: Dynamic Defense Against Large Language Model Jailbreaking with Model Editingcited by 9
  11. 2025SafeSteer: Interpretable Safety Steering with Refusal-Evasion in LLMscited by 9
  12. 2025Rethinking Jailbreak Detection of Large Vision Language Models with Representational Contrastive Scoringcited by 9
  13. 2024Recent advancements in LLM Red-Teaming: Techniques, Defenses, and Ethical Considerationscited by 8
  14. 2024A Middle Path for On-Premises LLM Deployment: Preserving Privacy Without Sacrificing Model Confidentialitycited by 8
  15. 2025Improving Large Language Model Safety with Contrastive Representation Learningcited by 7
  16. 2025Unraveling LLM Jailbreaks Through Safety Knowledge Neuronscited by 7
  17. 2025Layer-Level Self-Exposure and Patch: Affirmative Token Mitigation for Jailbreak Attack Defensecited by 6
  18. 2025Zero-Shot Defense Against Toxic Images via Inherent Multimodal Alignment in LVLMscited by 6
  19. 2025Beyond Black-Box Obfuscation: Mechanistic Analysis and Defense of White-Box Monitorscited by 6
  20. 2025From ASR to ASP: Evaluating Prompt Attack Vulnerabilities Against Open-Source LLMscited by 6
  21. 2025Disentangled Safety Adapters Enable Efficient Guardrails and Flexible Inference-Time Alignmentcited by 6
  22. 2024Prefix Guidance: A Steering Wheel for Large Language Models to Defend Against Jailbreak Attackscited by 5
  23. 2025Layer-Aware Task Arithmetic: Disentangling Task-Specific and Instruction-Following Knowledgecited by 5
  24. 2025Benchmarking Large Language Models for Cryptanalysis and Side-Channel Vulnerabilitiescited by 5
  25. 2025Large Language Model Sourcing: A Surveycited by 5
  26. 2026Now You Hear Me: Audio Narrative Attacks Against Large Audio-Language Modelscited by 5
  27. 2025SafeInt: Shielding Large Language Models from Jailbreak Attacks via Safety-Aware Representation Interventioncited by 4
  28. 2025Spectral Insights into Data-Oblivious Critical Layers in Large Language Modelscited by 4
  29. 2025CAVGAN: Unifying Jailbreak and Defense of LLMs via Generative Adversarial Attacks on their Internal Representationscited by 4
  30. 2025SoK: Systematizing LLM Prompt Security: Taxonomies, Datasets, and Unified Evaluation of Attacks and Defensescited by 4
  31. 2025STShield: Single-Token Sentinel for Real-Time Jailbreak Detection in Large Language Modelscited by 3
  32. 2026ALERT: Zero-shot LLM Jailbreak Detection via Internal Discrepancy Amplificationcited by 3
  33. 2026Large Language Models Generate Harmful Responses Using a Distinct Mechanism, Shared Across Harm Typescited by 3
  34. 2026MT-JailBench: A Modular Benchmark for Understanding Multi-Turn Jailbreak Attackscited by 3
  35. 2025PUZZLED: Jailbreaking LLMs through Word-Based Puzzlescited by 2
  36. 2025Q-MLLM: Vector Quantization for Robust Multimodal Large Language Model Securitycited by 2
  37. 2026Multi-turn Jailbreaking Attack in Multi-Modal Large Language Modelscited by 2
  38. 2026Layer-wise Swapping for Generalizable Multilingual Safetycited by 2

50 most cited of 77 All 77 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Jailbreak Attacks and Defenses Against Large Language Models: A Survey
  2. 2026Circuit Discovery Helps Detect LLM Jailbreaking: A Mechanistic Interpretability Study
  3. 2026Defending Large Language Models Against Jailbreak Attacks via In-Decoding Safety-Awareness Probing
  4. 2025PandaGuard: Systematic Evaluation of LLM Safety against Jailbreaking Attacks
  5. 2025Paper Summary Attack: Jailbreaking LLMs through LLM Safety Papers
  6. 2025Do Methods to Jailbreak and Defend LLMs Generalize Across Languages?
  7. 2025Scalable Defense against In-the-wild Jailbreaking Attacks with Safety Context Retrieval
  8. 2026Jailbreaking Leaves a Trace: Understanding and Detecting Jailbreak Attacks from Internal Representations of Large Language Models