mlsanityData to 7 Oct 2026

Defending Against Alignment-Breaking Attacks via Robustly Aligned LLM

arXiv 2309.14348 ↗First posted 18 Sep 2023cs.CL cs.AI cs.CRcited by 173

On the map: LLM alignment / RLHF / preference optimisation → Safety alignment and jailbreak attacks

also in 1 more
Abstract (arXiv)

Recently, Large Language Models (LLMs) have made significant advancements and are now widely used across various domains. Unfortunately, there has been a rising concern that LLMs can be misused to generate harmful or malicious content. Though a line of research has focused on aligning LLMs with human values and preventing them from producing inappropriate content, such alignments are usually vulnerable and can be bypassed by alignment-breaking attacks via adversarially optimized or handcrafted jailbreaking prompts. In this work, we introduce a Robustly Aligned LLM (RA-LLM) to defend against potential alignment-breaking attacks. RA-LLM can be directly constructed upon an existing aligned LLM with a robust alignment checking function, without requiring any expensive retraining or fine-tuning process of the original LLM. Furthermore, we also provide a theoretical analysis for RA-LLM to verify its effectiveness in defending against alignment-breaking attacks. Through real-world experiments on open-source large language models, we demonstrate that RA-LLM can successfully defend against both state-of-the-art adversarial prompts and popular handcrafted jailbreaking prompts by reducing their attack success rates from nearly 100% to around 10% or less.

Key facts. Cited by 173 papers in the map (data through 7 Oct 2026). Placed in Safety alignment and jailbreak attacks, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 39 of its 53 references that are in the map. First posted on arXiv 18 Sep 2023.

Builds on

39 of 53 references.

Closest ancestors · 3

Same direction, newest first

  1. 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Models1 Sep
  2. 2023GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Cipher12 Aug
  3. 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul

Other references · 36

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  5. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  6. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  7. 2017Deep reinforcement learning from human preferencescited by 3,846
  8. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  9. 2020Learning to summarize from human feedbackcited by 2,562
  10. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,591
  11. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
  12. 2019Certified Adversarial Robustness via Randomized Smoothingcited by 1,324
24 more
  1. 2019CTRL: A Conditional Transformer Language Model for Controllable Generationcited by 754
  2. 2019Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailmentcited by 663
  3. 2023XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Modelscited by 477
  4. 2023"Do Anything Now": Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Modelscited by 457
  5. 2018Generating Natural Language Adversarial Examplescited by 422
  6. 2018TextBugger: Generating Adversarial Text Against Real-world Applicationscited by 376
  7. 2018Black-box Generation of Adversarial Text Sequences to Evade Deep Learning Classifierscited by 341
  8. 2023Multi-step Jailbreaking Privacy Attacks on ChatGPTcited by 320
  9. 2016Adversarial Training Methods for Semi-Supervised Text Classificationcited by 280
  10. 2023Exploiting Programmatic Behavior of LLMs: Dual-Use Through Standard Security Attackscited by 268
  11. 2019FreeLB: Enhanced Adversarial Training for Natural Language Understandingcited by 261
  12. 2023Certifying LLM Safety against Adversarial Promptingcited by 248
  13. 2023Pretraining Language Models with Human Preferencescited by 245
  14. 2019Adversarial Attacks and Defenses in Images, Graphs and Text: A Reviewcited by 234
  15. 2023LLM Self Defense: By Self Examination, LLMs Know They Are Being Trickedcited by 226
  16. 2020Recipes for Safety in Open-domain Chatbotscited by 168
  17. 2019Certified Robustness to Adversarial Word Substitutionscited by 164
  18. 2023Fundamental Limitations of Alignment in Large Language Modelscited by 152
  19. 2023Aligning Language Models with Preferences through f-divergence Minimizationcited by 118
  20. 2020Robustness Verification for Transformerscited by 76
  21. 2021Towards Robustness Against Natural Language Word Substitutionscited by 68
  22. 2021Certified Robustness to Text Adversarial Attacks by Randomized [MASK]cited by 57
  23. 2020Generating Natural Language Attacks in a Hard Label Black Box Settingcited by 50
  24. 2023A Brief Report on LawGPT 1.0: A Virtual Legal Assistant Based on GPT-3cited by 46

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 16; 2024: 74; 2025: 61; 2026: 22.

173 papers.

Sort:

By area:

  1. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
  2. 2024How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMscited by 533
  3. 2023SmoothLLM: Defending Large Language Models Against Jailbreaking Attackscited by 434
  4. 2023Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrationscited by 401
  5. 2023A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Uglycited by 380
  6. 2023Shadow Alignment: The Ease of Subverting Safely-Aligned Language Modelscited by 284
  7. 2023MM-SafetyBench: A Benchmark for Safety Evaluation of Multimodal Large Language Modelscited by 255
  8. 2023A Wolf in Sheep's Clothing: Generalized Nested Jailbreak Prompts can Fool Large Language Models Easilycited by 242
  9. 2024SafeDecoding: Defending against Jailbreak Attacks via Safety-Aware Decodingcited by 233
  10. 2024SALAD-Bench: A Hierarchical and Comprehensive Safety Benchmark for Large Language Modelscited by 225
  11. 2024Jailbreak Attacks and Defenses Against Large Language Models: A Surveycited by 215
  12. 2023Defending Large Language Models Against Jailbreaking Attacks Through Goal Prioritizationcited by 156
38 more
  1. 2024Robust Prompt Optimization for Defending Language Models Against Jailbreaking Attackscited by 150
  2. 2024AutoDefense: Multi-Agent LLM Defense against Jailbreak Attackscited by 135
  3. 2024Know Your Limits: A Survey of Abstention in Large Language Modelscited by 116
  4. 2024Vaccine: Perturbation-aware Alignment for Large Language Models against Harmful Fine-tuning Attackcited by 113
  5. 2024Personalized Steering of Large Language Models: Versatile Steering Vectors Through Bi-directional Preference Optimizationcited by 112
  6. 2023Evil Geniuses: Delving into the Safety of LLM-based Agentscited by 103
  7. 2023AutoDAN: Interpretable Gradient-Based Adversarial Attacks on Large Language Modelscited by 102
  8. 2024Weak-to-Strong Jailbreaking on Large Language Modelscited by 100
  9. 2024A Comprehensive Study of Jailbreak Attack versus Defense for Large Language Modelscited by 95
  10. 2024Making Them Ask and Answer: Jailbreaking Large Language Models in Few Queries via Disguise and Reconstructioncited by 95
  11. 2024DrAttack: Prompt Decomposition and Reconstruction Makes Powerful LLM Jailbreakerscited by 91
  12. 2024Detoxifying Large Language Models via Knowledge Editingcited by 86
  13. 2024A Survey of Large Language Models for Financial Applications: Progress, Prospects and Challengescited by 86
  14. 2024Harmful Fine-tuning Attacks and Defenses for Large Language Models: A Surveycited by 86
  15. 2024Red-Teaming for Generative AI: Silver Bullet or Security Theater?cited by 81
  16. 2024LLMs know their vulnerabilities: Uncover Safety Gaps through Natural Distribution Shiftscited by 80
  17. 2024PsySafe: A Comprehensive Framework for Psychological-based Attack, Defense, and Evaluation of Multi-agent System Safetycited by 79
  18. 2023Cognitive Overload: Jailbreaking Large Language Models with Overloaded Logical Thinkingcited by 78
  19. 2024Defending Large Language Models Against Jailbreak Attacks via Layer-specific Editingcited by 77
  20. 2024Safeguarding Large Language Models: A Surveycited by 73
  21. 2024Defending Large Language Models against Jailbreak Attacks via Semantic Smoothingcited by 68
  22. 2024Improved Few-Shot Jailbreaking Can Circumvent Aligned Language Models and Their Defensescited by 68
  23. 2024JailbreakZoo: Survey, Landscapes, and Horizons in Jailbreaking Large Language and Vision-Language Modelscited by 67
  24. 2024Refuse Whenever You Feel Unsafe: Improving Safety in LLMs via Decoupled Refusal Trainingcited by 67
  25. 2024RigorLLM: Resilient Guardrails for Large Language Models against Undesired Contentcited by 65
  26. 2024Intention Analysis Makes LLMs A Good Jailbreak Defendercited by 64
  27. 2023The Art of Defending: A Systematic Evaluation and Analysis of LLM Defense Strategies on Safety and Over-Defensivenesscited by 63
  28. 2024Fight Back Against Jailbreaking via Prompt Adversarial Tuningcited by 63
  29. 2024Robust LLM safeguarding via refusal feature adversarial trainingcited by 60
  30. 2024EasyJailbreak: A Unified Framework for Jailbreaking Large Language Modelscited by 57
  31. 2024SelfDefend: LLMs Can Defend Themselves against Jailbreaking in a Practical Mannercited by 57
  32. 2024Context-DPO: Aligning Language Models for Context-Faithfulnesscited by 50
  33. 2024Risks of AI Scientists: Prioritizing Safeguarding Over Autonomycited by 49
  34. 2024Eraser: Jailbreaking Defense in Large Language Models via Unlearning Harmful Knowledgecited by 48
  35. 2024Bag of Tricks: Benchmarking of Jailbreak Attacks on LLMscited by 47
  36. 2024JAILJUDGE: A Comprehensive Jailbreak Judge Benchmark with Multi-Agent Enhanced Explanation Evaluation Frameworkcited by 44
  37. 2024A Theoretical Understanding of Self-Correction through In-context Alignmentcited by 43
  38. 2024Boosting Jailbreak Attack with Momentumcited by 38

50 most cited of 173 All 173 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025AlignTree: Efficient Defense Against LLM Jailbreak Attacks
  2. 2025Targeting Alignment: Extracting Safety Classifiers of Aligned LLMs
  3. 2024Jailbreaking Large Language Models Through Alignment Vulnerabilities in Out-of-Distribution Settings
  4. 2024Safety Alignment Should Be Made More Than Just a Few Tokens Deep
  5. 2025Scaling Patterns in Adversarial Alignment: Evidence from Multi-LLM Jailbreak Experiments
  6. 2025HAMSA: Hijacking Aligned Compact Models via Stealthy Automation
  7. 2023Catastrophic Jailbreak of Open-source LLMs via Exploiting Generation
  8. 2025LLM Reinforcement in Context