mlsanityData to 7 Oct 2026

SafeDecoding: Defending against Jailbreak Attacks via Safety-Aware Decoding

arXiv 2402.08983 ↗First posted 14 Feb 2024cs.CR cs.AI cs.CLcited by 233

On the map: LLM safety and security → Jailbreaking

Abstract (arXiv)

As large language models (LLMs) become increasingly integrated into real-world applications such as code generation and chatbot assistance, extensive efforts have been made to align LLM behavior with human values, including safety. Jailbreak attacks, aiming to provoke unintended and unsafe behaviors from LLMs, remain a significant/leading LLM safety threat. In this paper, we aim to defend LLMs against jailbreak attacks by introducing SafeDecoding, a safety-aware decoding strategy for LLMs to generate helpful and harmless responses to user queries. Our insight in developing SafeDecoding is based on the observation that, even though probabilities of tokens representing harmful contents outweigh those representing harmless responses, safety disclaimers still appear among the top tokens after sorting tokens by probability in descending order. This allows us to mitigate jailbreak attacks by identifying safety disclaimers and amplifying their token probabilities, while simultaneously attenuating the probabilities of token sequences that are aligned with the objectives of jailbreak attacks. We perform extensive experiments on five LLMs using six state-of-the-art jailbreak attacks and four benchmark datasets. Our results show that SafeDecoding significantly reduces the attack success rate and harmfulness of jailbreak attacks without compromising the helpfulness of responses to benign user queries. SafeDecoding outperforms six defense methods.

Key facts. Cited by 233 papers in the map (data through 7 Oct 2026). Placed in Jailbreaking, one of 12 directions in LLM safety and security. Builds on 40 of its 48 references that are in the map. First posted on arXiv 14 Feb 2024.

Builds on

40 of 48 references.

Closest ancestors · 14

Same direction, newest first

  1. 2024Robust Prompt Optimization for Defending Language Models Against Jailbreaking Attacks30 Jan
  2. 2024How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMs12 Jan
  3. 2023Defending Large Language Models Against Jailbreaking Attacks Through Goal Prioritization15 Nov
  4. 2023DeepInception: Hypnotize Large Language Model to Be Jailbreaker6 Nov
  5. 2023Jailbreaking Black Box Large Language Models in Twenty Queries12 Oct
  6. 2023Catastrophic Jailbreak of Open-source LLMs via Exploiting Generation10 Oct
  7. 2023Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrations10 Oct
  8. 2023SmoothLLM: Defending Large Language Models Against Jailbreaking Attacks5 Oct
  9. 2023GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Prompts19 Sep
  10. 2023Defending Against Alignment-Breaking Attacks via Robustly Aligned LLM18 Sep
  11. 2023Detecting Language Model Attacks with Perplexity27 Aug
  12. 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul
2 more
  1. 2023Jailbroken: How Does LLM Safety Training Fail?5 Jul
  2. 2023Jailbreaking ChatGPT via Prompt Engineering: An Empirical Study23 May

Other references · 26

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  5. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  6. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  7. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  8. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  9. 2016Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translationcited by 1,471
  10. 2018Hierarchical Neural Story Generationcited by 1,182
  11. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,106
  12. 2023LIMA: Less Is More for Alignmentcited by 1,080
14 more
  1. 2021Ethical and social risks of harm from Language Modelscited by 957
  2. 2023The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Onlycited by 755
  3. 2022Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learnedcited by 702
  4. 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Modelscited by 608
  5. 2022Contrastive Decoding: Open-ended Text Generation as Optimizationcited by 598
  6. 2022Improving alignment of dialogue agents via targeted human judgementscited by 527
  7. 2021Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Surveycited by 457
  8. 2023The Unlocking Spell on Base LLMs: Rethinking Alignment via In-Context Learningcited by 251
  9. 2023LLM Self Defense: By Self Examination, LLMs Know They Are Being Trickedcited by 226
  10. 2023Multimodal Large Language Models: A Surveycited by 193
  11. 2019BPE-Dropout: Simple and Effective Subword Regularizationcited by 189
  12. 2023Automatically Auditing Large Language Models via Discrete Optimizationcited by 187
  13. 2023RAIN: Your Language Models Can Align Themselves without Finetuningcited by 149
  14. 2023Identifying and Mitigating Vulnerabilities in LLM-Integrated Applicationscited by 54

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 63; 2025: 105; 2026: 65.

233 papers.

Sort:

By area:

  1. 2024Safety Alignment Should Be Made More Than Just a Few Tokens Deepcited by 450
  2. 2024Jailbreak Attacks and Defenses Against Large Language Models: A Surveycited by 215
  3. 2024ArtPrompt: ASCII Art-based Jailbreak Attacks against Aligned LLMscited by 197
  4. 2024VoiceBench: Benchmarking LLM-Based Voice Assistantscited by 192
  5. 2024AmpleGCG: Learning a Universal and Transferable Generative Model of Adversarial Suffixes for Jailbreaking Both Open and Closed LLMscited by 160
  6. 2025SafeChain: Safety of Language Models with Long Chain-of-Thought Reasoning Capabilitiescited by 133
  7. 2024LLMs know their vulnerabilities: Uncover Safety Gaps through Natural Distribution Shiftscited by 80
  8. 2024Defending Large Language Models Against Jailbreak Attacks via Layer-specific Editingcited by 77
  9. 2024Safeguarding Large Language Models: A Surveycited by 73
  10. 2024FlipAttack: Jailbreak LLMs via Flippingcited by 73
  11. 2024Improved Few-Shot Jailbreaking Can Circumvent Aligned Language Models and Their Defensescited by 68
  12. 2024JailbreakZoo: Survey, Landscapes, and Horizons in Jailbreaking Large Language and Vision-Language Modelscited by 67
38 more
  1. 2024Refuse Whenever You Feel Unsafe: Improving Safety in LLMs via Decoupled Refusal Trainingcited by 67
  2. 2024Gradient Cuff: Detecting Jailbreak Attacks on Large Language Models by Exploring Refusal Loss Landscapescited by 63
  3. 2024Navigating the Risks: A Survey of Security, Privacy, and Ethics Threats in LLM-Based Agentscited by 63
  4. 2024SelfDefend: LLMs Can Defend Themselves against Jailbreaking in a Practical Mannercited by 57
  5. 2024BEEAR: Embedding-based Adversarial Removal of Safety Backdoors in Instruction-tuned Language Modelscited by 50
  6. 2024Surgical, Cheap, and Flexible: Mitigating False Refusal in Language Models via Single Vector Ablationcited by 48
  7. 2024SCANS: Mitigating the Exaggerated Safety for LLMs via Safety-Conscious Activation Steeringcited by 45
  8. 2024Operationalizing a Threat Model for Red-Teaming Large Language Models (LLMs)cited by 44
  9. 2025SciReplicate-Bench: Benchmarking LLMs in Agent-driven Algorithmic Reproduction from Research Paperscited by 42
  10. 2025SaLoRA: Safety-Alignment Preserved Low-Rank Adaptationcited by 39
  11. 2024Boosting Jailbreak Attack with Momentumcited by 38
  12. 2025The Hidden Dimensions of LLM Alignment: A Multi-Dimensional Analysis of Orthogonal Safety Directionscited by 38
  13. 2025AdaSteer: Your Aligned LLM is Inherently an Adaptive Jailbreak Defendercited by 37
  14. 2025Reasoning-to-Defend: Safety-Aware Reasoning Can Defend Large Language Models from Jailbreakingcited by 34
  15. 2024SHIELD: Evaluation and Defense Strategies for Copyright Compliance in LLM Text Generationcited by 33
  16. 2024Jailbreaking and Mitigation of Vulnerabilities in Large Language Modelscited by 33
  17. 2024Safety Arithmetic: A Framework for Test-time Safety Alignment of Language Models by Steering Parameters and Activationscited by 32
  18. 2025The Devil behind the mask: An emergent safety vulnerability of Diffusion LLMscited by 32
  19. 2024ChatBug: A Common Vulnerability of Aligned LLMs Induced by Chat Templatescited by 29
  20. 2024No Two Devils Alike: Unveiling Distinct Mechanisms of Fine-tuning Attackscited by 28
  21. 2024JailbreakEval: An Integrated Toolkit for Evaluating Jailbreak Attempts Against Large Language Modelscited by 28
  22. 2024BlueSuffix: Reinforced Blue Teaming for Vision-Language Models Against Jailbreak Attackscited by 28
  23. 2024Shaping the Safety Boundaries: Understanding and Defending Against Jailbreaks in Large Language Modelscited by 23
  24. 2025AdaptiveStep: Automatically Dividing Reasoning Step through Model Confidencecited by 23
  25. 2025RAG LLMs are Not Safer: A Safety Analysis of Retrieval-Augmented Generation for Large Language Modelscited by 23
  26. 2024SafeInfer: Context Adaptive Decoding Time Safety Alignment for Large Language Modelscited by 22
  27. 2024From LLMs to MLLMs: Exploring the Landscape of Multimodal Jailbreakingcited by 22
  28. 2025Just Enough Shifts: Mitigating Over-Refusal in Aligned Language Models with Targeted Representation Fine-Tuningcited by 22
  29. 2025A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Modelscited by 22
  30. 2024Mission Impossible: A Statistical Perspective on Jailbreaking LLMscited by 21
  31. 2024On Calibration of LLM-based Guard Models for Reliable Content Moderationcited by 21
  32. 2024CleanGen: Mitigating Backdoor Attacks for Generation Tasks in Large Language Modelscited by 20
  33. 2024PAPILLON: Efficient and Stealthy Fuzz Testing-Powered Jailbreaks for LLMscited by 20
  34. 2024Separate the Wheat from the Chaff: A Post-Hoc Approach to Safety Re-Alignment for Fine-Tuned Language Modelscited by 19
  35. 2025SoK: Evaluating Jailbreak Guardrails for Large Language Modelscited by 17
  36. 2024Towards Comprehensive Post Safety Alignment of Large Language Models via Safety Patchingcited by 16
  37. 2025QueryAttack: Jailbreaking Aligned Large Language Models Using Structured Non-natural Query Languagecited by 16
  38. 2025Advancing LLM Safe Alignment with Safety Representation Rankingcited by 15

50 most cited of 233 All 233 (JSON)