mlsanityData to 7 Oct 2026

How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMs

arXiv 2401.06373 ↗First posted 12 Jan 2024cs.CL cs.AIcited by 533

On the map: LLM reasoning → Reasoning for agents, planning and code

also in 1 more
Abstract (arXiv)

Most traditional AI safety research has approached AI models as machines and centered on algorithm-focused attacks developed by security experts. As large language models (LLMs) become increasingly common and competent, non-expert users can also impose risks during daily interactions. This paper introduces a new perspective to jailbreak LLMs as human-like communicators, to explore this overlooked intersection between everyday language interaction and AI safety. Specifically, we study how to persuade LLMs to jailbreak them. First, we propose a persuasion taxonomy derived from decades of social science research. Then, we apply the taxonomy to automatically generate interpretable persuasive adversarial prompts (PAP) to jailbreak LLMs. Results show that persuasion significantly increases the jailbreak performance across all risk categories: PAP consistently achieves an attack success rate of over 92% on Llama 2-7b Chat, GPT-3.5, and GPT-4 in 10 trials, surpassing recent algorithm-focused attacks. On the defense side, we explore various mechanisms against PAP and, found a significant gap in existing defenses, and advocate for more fundamental mitigation for highly interactive LLMs

Key facts. Cited by 533 papers in the map (data through 7 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 35 of its 70 references that are in the map. First posted on arXiv 12 Jan 2024.

Builds on

35 of 70 references.

References · 35

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  4. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  5. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,326
  6. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,235
  7. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,106
  8. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Modelscited by 738
  9. 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Modelscited by 608
  10. 2023GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Promptscited by 520
  11. 2023Jailbreaking ChatGPT via Prompt Engineering: An Empirical Studycited by 478
  12. 2023Catastrophic Jailbreak of Open-source LLMs via Exploiting Generationcited by 439
23 more
  1. 2023SmoothLLM: Defending Large Language Models Against Jailbreaking Attackscited by 434
  2. 2023Detecting Language Model Attacks with Perplexitycited by 411
  3. 2023GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Ciphercited by 401
  4. 2023Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrationscited by 401
  5. 2023Are aligned neural networks adversarially aligned?cited by 305
  6. 2023Low-Resource Languages Jailbreak GPT-4cited by 290
  7. 2023Shadow Alignment: The Ease of Subverting Safely-Aligned Language Modelscited by 284
  8. 2023Exploiting Programmatic Behavior of LLMs: Dual-Use Through Standard Security Attackscited by 268
  9. 2023Certifying LLM Safety against Adversarial Promptingcited by 248
  10. 2021Gradient-based Adversarial Attacks against Text Transformerscited by 233
  11. 2023Multilingual Jailbreak Challenges in Large Language Modelscited by 222
  12. 2019Persuasion for Good: Towards a Personalized Persuasive Dialogue System for Social Goodcited by 207
  13. 2023Scalable and Transferable Black-Box Jailbreaks for Language Models via Persona Modulationcited by 202
  14. 2023Automatically Auditing Large Language Models via Discrete Optimizationcited by 187
  15. 2023Defending Against Alignment-Breaking Attacks via Robustly Aligned LLMcited by 173
  16. 2023MasterKey: Automated Jailbreak Across Multiple Large Language Model Chatbotscited by 172
  17. 2023RAIN: Your Language Models Can Align Themselves without Finetuningcited by 149
  18. 2023What's In My Big Data?cited by 129
  19. 2023Open Sesame! Universal Black Box Jailbreaking of Large Language Modelscited by 128
  20. 2023The Earth is Flat because...: Investigating LLMs' Belief towards Misinformation via Persuasive Conversationcited by 107
  21. 2023Use of LLMs for Illicit Purposes: Threats, Prevention Measures, and Vulnerabilitiescited by 76
  22. 2023Adversarial Demonstration Attacks on Large Language Modelscited by 58
  23. 2023Susceptibility to Influence of Large Language Modelscited by 17

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 177; 2025: 221; 2026: 135.

533 papers.

Sort:

By area:

  1. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
  2. 2024JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Modelscited by 494
  3. 2024Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attackscited by 450
  4. 2024Safety Alignment Should Be Made More Than Just a Few Tokens Deepcited by 450
  5. 2024A StrongREJECT for Empty Jailbreakscited by 352
  6. 2024Improving Alignment and Robustness with Circuit Breakerscited by 302
  7. 2024SafeDecoding: Defending against Jailbreak Attacks via Safety-Aware Decodingcited by 233
  8. 2024SORRY-Bench: Systematically Evaluating Large Language Model Safety Refusalcited by 224
  9. 2024OR-Bench: An Over-Refusal Benchmark for Large Language Modelscited by 222
  10. 2024Assessing the Brittleness of Safety Alignment via Pruning and Low-Rank Modificationscited by 220
  11. 2024Jailbreak Attacks and Defenses Against Large Language Models: A Surveycited by 215
  12. 2024AutoDAN-Turbo: A Lifelong Agent for Strategy Self-Exploration to Jailbreak LLMscited by 185
38 more
  1. 2024COLD-Attack: Jailbreaking LLMs with Stealthiness and Controllabilitycited by 162
  2. 2024AmpleGCG: Learning a Universal and Transferable Generative Model of Adversarial Suffixes for Jailbreaking Both Open and Closed LLMscited by 160
  3. 2024Robust Prompt Optimization for Defending Language Models Against Jailbreaking Attackscited by 150
  4. 2024AdvPrompter: Fast Adaptive Adversarial Prompting for LLMscited by 146
  5. 2024LLM Defenses Are Not Robust to Multi-Turn Human Jailbreaks Yetcited by 145
  6. 2024Is LLM-as-a-Judge Robust? Investigating Universal Adversarial Attacks on Zero-shot LLM Assessmentcited by 117
  7. 2024What is in Your Safe Data? Identifying Benign Data that Breaks Safetycited by 114
  8. 2024Attacking Vision-Language Computer Agents via Pop-upscited by 112
  9. 2024On Prompt-Driven Safeguarding for Large Language Modelscited by 109
  10. 2024Weak-to-Strong Jailbreaking on Large Language Modelscited by 100
  11. 2024Improved Techniques for Optimization-Based Jailbreaking on Large Language Modelscited by 98
  12. 2024Making Them Ask and Answer: Jailbreaking Large Language Models in Few Queries via Disguise and Reconstructioncited by 95
  13. 2024Keeping LLMs Aligned After Fine-tuning: The Crucial Role of Prompt Templatescited by 94
  14. 2024Coercing LLMs to do and reveal (almost) anythingcited by 88
  15. 2024LLMs know their vulnerabilities: Uncover Safety Gaps through Natural Distribution Shiftscited by 80
  16. 2024Defending Large Language Models Against Jailbreak Attacks via Layer-specific Editingcited by 77
  17. 2025LLMs Encode Harmfulness and Refusal Separatelycited by 75
  18. 2025STAIR: Improving Safety Alignment with Introspective Reasoningcited by 72
  19. 2024Best-of-N Jailbreakingcited by 71
  20. 2024Can LLMs Separate Instructions From Data? And What Do We Even Mean By That?cited by 70
  21. 2024CodeAttack: Revealing Safety Generalization Challenges of Large Language Models via Code Completioncited by 70
  22. 2024On the Role of Attention Heads in Large Language Model Safetycited by 70
  23. 2024AIR-Bench 2024: A Safety Benchmark Based on Risk Categories from Regulations and Policiescited by 69
  24. 2024Defending Large Language Models against Jailbreak Attacks via Semantic Smoothingcited by 68
  25. 2024Improved Few-Shot Jailbreaking Can Circumvent Aligned Language Models and Their Defensescited by 68
  26. 2024Refuse Whenever You Feel Unsafe: Improving Safety in LLMs via Decoupled Refusal Trainingcited by 67
  27. 2024Defending LLMs against Jailbreaking Attacks via Backtranslationcited by 65
  28. 2024RigorLLM: Resilient Guardrails for Large Language Models against Undesired Contentcited by 65
  29. 2024Visual-RolePlay: Universal Jailbreak Attack on MultiModal Large Language Models via Role-playing Image Charactercited by 61
  30. 2024Refusal-Trained LLMs Are Easily Jailbroken As Browser Agentscited by 61
  31. 2025Reasoning-Augmented Conversation for Multi-Turn Jailbreak Attacks on Large Language Modelscited by 61
  32. 2024"Glue pizza and eat rocks" -- Exploiting Vulnerabilities in Retrieval-Augmented Generative Modelscited by 60
  33. 2025Safety in Large Reasoning Models: A Surveycited by 58
  34. 2024EasyJailbreak: A Unified Framework for Jailbreaking Large Language Modelscited by 57
  35. 2024Jailbreak Large Vision-Language Models Through Multi-Modal Linkagecited by 53
  36. 2024On Evaluating the Durability of Safeguards for Open-Weight LLMscited by 51
  37. 2024BEEAR: Embedding-based Adversarial Removal of Safety Backdoors in Instruction-tuned Language Modelscited by 50
  38. 2024PRP: Propagating Universal Perturbations to Attack Large Language Model Guard-Railscited by 49

50 most cited of 533 All 533 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Intention Analysis Makes LLMs A Good Jailbreak Defender
  2. 2025Automating Deception: Scalable Multi-Turn LLM Jailbreaks
  3. 2025Speak Easy: Eliciting Harmful Jailbreaks from LLMs with Simple Interactions
  4. 2026Circuit Discovery Helps Detect LLM Jailbreaking: A Mechanistic Interpretability Study
  5. 2025Do Internal Layers of LLMs Reveal Patterns for Jailbreak Detection?
  6. 2025PandaGuard: Systematic Evaluation of LLM Safety against Jailbreaking Attacks
  7. 2024Defending against Jailbreak through Early Exit Generation of Large Language Models
  8. 2026SoK: Rethinking Jailbreaking in the Era of Agentic AI: Attacks, Defenses, and Practical Consideration