mlsanityData to 7 Oct 2026

SmoothLLM: Defending Large Language Models Against Jailbreaking Attacks

arXiv 2310.03684 ↗First posted 5 Oct 2023cs.LG cs.AI stat.MLcited by 434

On the map: LLM safety and security → Jailbreaking

Abstract (arXiv)

Despite efforts to align large language models (LLMs) with human intentions, widely-used LLMs such as GPT, Llama, and Claude are susceptible to jailbreaking attacks, wherein an adversary fools a targeted LLM into generating objectionable content. To address this vulnerability, we propose SmoothLLM, the first algorithm designed to mitigate jailbreaking attacks. Based on our finding that adversarially-generated prompts are brittle to character-level changes, our defense randomly perturbs multiple copies of a given input prompt, and then aggregates the corresponding predictions to detect adversarial inputs. Across a range of popular LLMs, SmoothLLM sets the state-of-the-art for robustness against the GCG, PAIR, RandomSearch, and AmpleGCG jailbreaks. SmoothLLM is also resistant against adaptive GCG attacks, exhibits a small, though non-negligible trade-off between robustness and nominal performance, and is compatible with any LLM. Our code is publicly available at urlhttps://github.com/arobey1/smooth-llm.

Key facts. Cited by 434 papers in the map (data through 7 Oct 2026). Placed in Jailbreaking, one of 12 directions in LLM safety and security. Builds on 75 of its 131 references that are in the map. First posted on arXiv 5 Oct 2023.

Builds on

75 of 131 references.

Closest ancestors · 9

Same direction, newest first

  1. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models3 Oct
  2. 2023Low-Resource Languages Jailbreak GPT-43 Oct
  3. 2023GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Prompts19 Sep
  4. 2023Defending Against Alignment-Breaking Attacks via Robustly Aligned LLM18 Sep
  5. 2023Certifying LLM Safety against Adversarial Prompting6 Sep
  6. 2023Detecting Language Model Attacks with Perplexity27 Aug
  7. 2023"Do Anything Now": Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Models7 Aug
  8. 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul
  9. 2023Jailbroken: How Does LLM Safety Training Fail?5 Jul

Other references · 66

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  4. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  5. 2013Intriguing properties of neural networkscited by 5,114
  6. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,818
  7. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,114
  8. 2019Invariant Risk Minimizationcited by 1,832
  9. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,510
  10. 2020WILDS: A Benchmark of in-the-Wild Distribution Shiftscited by 1,366
  11. 2019Certified Adversarial Robustness via Randomized Smoothingcited by 1,324
  12. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,239
38 more
  1. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,235
  2. 2023BloombergGPT: A Large Language Model for Financecited by 923
  3. 2018Robustness May Be at Odds with Accuracycited by 873
  4. 2020Fast is better than free: Revisiting adversarial trainingcited by 723
  5. 2017Evasion Attacks against Machine Learning at Test Timecited by 686
  6. 2020RobustBench: a standardized adversarial robustness benchmarkcited by 659
  7. 2019Adversarial Training for Free!cited by 638
  8. 2021Efficient Large-Scale Language Model Training on GPU Clusters Using Megatron-LMcited by 621
  9. 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Modelscited by 608
  10. 2017Provable defenses against adversarial examples via the convex outer adversarial polytopecited by 564
  11. 2020On Adaptive Attacks to Adversarial Example Defensescited by 547
  12. 2022Improving alignment of dialogue agents via targeted human judgementscited by 527
  13. 2018Certified Robustness to Adversarial Examples with Differential Privacycited by 462
  14. 2018Generalizing to Unseen Domains via Adversarial Data Augmentationcited by 451
  15. 2020Artificial Intelligence, Values and Alignmentcited by 443
  16. 2018Generating Natural Language Adversarial Examplescited by 422
  17. 2022ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detectioncited by 419
  18. 2018TextBugger: Generating Adversarial Text Against Real-world Applicationscited by 376
  19. 2023Toxicity in ChatGPT: Analyzing Persona-assigned Language Modelscited by 356
  20. 2019Provably Robust Deep Learning via Adversarially Trained Smoothed Classifierscited by 328
  21. 2018Certified Defenses against Adversarial Examplescited by 305
  22. 2023Are aligned neural networks adversarially aligned?cited by 305
  23. 2019Efficient and Accurate Estimation of Lipschitz Constants for Deep Neural Networkscited by 290
  24. 2016Adversarial Training Methods for Semi-Supervised Text Classificationcited by 280
  25. 2023Red-Teaming Large Language Models using Chain of Utterances for Safety-Alignmentcited by 213
  26. 2019Combating Adversarial Misspellings with Robust Word Recognitioncited by 162
  27. 2020BREEDS: Benchmarks for Subpopulation Shiftcited by 155
  28. 2020Randomized Smoothing of All Shapes and Sizescited by 146
  29. 2020Perceptual Adversarial Robustness: Defense Against Unseen Threat Modelscited by 137
  30. 2022(Certified!!) Adversarial Robustness for Free!cited by 130
  31. 2020Maximum-Entropy Adversarial Data Augmentation for Improved Generalization and Robustnesscited by 121
  32. 2019Adversarial AutoAugmentcited by 111
  33. 2020Adversarial Training for Large Neural Language Modelscited by 109
  34. 2020Certified Robustness to Label-Flipping Attacks via Randomized Smoothingcited by 103
  35. 2021Model-Based Domain Generalizationcited by 103
  36. 2023Regulating ChatGPT and other Large Generative AI Modelscited by 102
  37. 2020(De)Randomized Smoothing for Certifiable Defense against Patch Attackscited by 97
  38. 2021AugMax: Adversarial Composition of Random Augmentations for Robust Trainingcited by 89

50 most cited of 66 All 66 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 16; 2024: 146; 2025: 160; 2026: 112.

434 papers.

Sort:

By area:

  1. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,326
  2. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 960
  3. 2024How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMscited by 533
  4. 2024JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Modelscited by 494
  5. 2024InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agentscited by 476
  6. 2024Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attackscited by 450
  7. 2023Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrationscited by 401
  8. 2023A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Uglycited by 380
  9. 2024A StrongREJECT for Empty Jailbreakscited by 352
  10. 2024Improving Alignment and Robustness with Circuit Breakerscited by 302
  11. 2024Personal LLM Agents: Insights and Survey about the Capability, Efficiency and Securitycited by 257
  12. 2024SafeDecoding: Defending against Jailbreak Attacks via Safety-Aware Decodingcited by 233
38 more
  1. 2024SALAD-Bench: A Hierarchical and Comprehensive Safety Benchmark for Large Language Modelscited by 225
  2. 2024OR-Bench: An Over-Refusal Benchmark for Large Language Modelscited by 222
  3. 2024Jailbreak Attacks and Defenses Against Large Language Models: A Surveycited by 215
  4. 2024ArtPrompt: ASCII Art-based Jailbreak Attacks against Aligned LLMscited by 197
  5. 2024Rainbow Teaming: Open-Ended Generation of Diverse Adversarial Promptscited by 189
  6. 2024Security and Privacy Challenges of Large Language Models: A Surveycited by 179
  7. 2023Defending Large Language Models Against Jailbreaking Attacks Through Goal Prioritizationcited by 156
  8. 2024Robust Prompt Optimization for Defending Language Models Against Jailbreaking Attackscited by 150
  9. 2024AutoDefense: Multi-Agent LLM Defense against Jailbreak Attackscited by 135
  10. 2024The Emerged Security and Privacy of LLM Agent: A Survey with Case Studiescited by 127
  11. 2025Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systemscited by 125
  12. 2024Attacks, Defenses and Evaluations for LLM Conversation Safety: A Surveycited by 124
  13. 2024Is LLM-as-a-Judge Robust? Investigating Universal Adversarial Attacks on Zero-shot LLM Assessmentcited by 117
  14. 2024Know Your Limits: A Survey of Abstention in Large Language Modelscited by 116
  15. 2024MLLM-Protector: Ensuring MLLM's Safety without Hurting Performancecited by 111
  16. 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 110
  17. 2024Efficient Adversarial Training in LLMs with Continuous Attackscited by 104
  18. 2024Jailbreak Vision Language Models via Bi-Modal Adversarial Promptcited by 103
  19. 2023AutoDAN: Interpretable Gradient-Based Adversarial Attacks on Large Language Modelscited by 102
  20. 2024A Comprehensive Study of Jailbreak Attack versus Defense for Large Language Modelscited by 95
  21. 2024AutoAttacker: A Large Language Model Guided System to Implement Automatic Cyber-attackscited by 94
  22. 2024Guardrail Baselines for Unlearning in LLMscited by 90
  23. 2023Jatmo: Prompt Injection Defense by Task-Specific Finetuningcited by 89
  24. 2024Coercing LLMs to do and reveal (almost) anythingcited by 88
  25. 2024Detoxifying Large Language Models via Knowledge Editingcited by 86
  26. 2024Harmful Fine-tuning Attacks and Defenses for Large Language Models: A Surveycited by 86
  27. 2024Red-Teaming for Generative AI: Silver Bullet or Security Theater?cited by 81
  28. 2024LLMs know their vulnerabilities: Uncover Safety Gaps through Natural Distribution Shiftscited by 80
  29. 2024PsySafe: A Comprehensive Framework for Psychological-based Attack, Defense, and Evaluation of Multi-agent System Safetycited by 79
  30. 2023Cognitive Overload: Jailbreaking Large Language Models with Overloaded Logical Thinkingcited by 78
  31. 2024Defending Large Language Models Against Jailbreak Attacks via Layer-specific Editingcited by 77
  32. 2024Safeguarding Large Language Models: A Surveycited by 73
  33. 2024FlipAttack: Jailbreak LLMs via Flippingcited by 73
  34. 2024CodeAttack: Revealing Safety Generalization Challenges of Large Language Models via Code Completioncited by 70
  35. 2024AIR-Bench 2024: A Safety Benchmark Based on Risk Categories from Regulations and Policiescited by 69
  36. 2024Defending Large Language Models against Jailbreak Attacks via Semantic Smoothingcited by 68
  37. 2024Improved Few-Shot Jailbreaking Can Circumvent Aligned Language Models and Their Defensescited by 68
  38. 2024JailbreakZoo: Survey, Landscapes, and Horizons in Jailbreaking Large Language and Vision-Language Modelscited by 67

50 most cited of 434 All 434 (JSON)