mlsanityData to 7 Oct 2026

DeepInception: Hypnotize Large Language Model to Be Jailbreaker

arXiv 2311.03191 ↗First posted 6 Nov 2023cs.LG cs.CRcited by 337

On the map: LLM safety and security → Jailbreaking

Abstract (arXiv)

Large language models (LLMs) have succeeded significantly in various applications but remain susceptible to adversarial jailbreaks that void their safety guardrails. Previous attempts to exploit these vulnerabilities often rely on high-cost computational extrapolations, which may not be practical or efficient. In this paper, inspired by the authority influence demonstrated in the Milgram experiment, we present a lightweight method to take advantage of the LLMs' personification capabilities to construct textita virtual, nested scene, allowing it to realize an adaptive way to escape the usage control in a normal scenario. Empirically, the contents induced by our approach can achieve leading harmfulness rates with previous counterparts and realize a continuous jailbreak in subsequent interactions, which reveals the critical weakness of self-losing on both open-source and closed-source LLMs, textite.g., Llama-2, Llama-3, GPT-3.5, GPT-4, and GPT-4o. The code and data are available at: https://github.com/tmlr-group/DeepInception.

Key facts. Cited by 337 papers in the map (data through 7 Oct 2026). Placed in Jailbreaking, one of 12 directions in LLM safety and security. Builds on 18 of its 26 references that are in the map. First posted on arXiv 6 Nov 2023.

Builds on

18 of 26 references.

Closest ancestors · 5

Same direction, newest first

  1. 2023Jailbreaking Black Box Large Language Models in Twenty Queries12 Oct
  2. 2023Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrations10 Oct
  3. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models3 Oct
  4. 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul
  5. 2023Jailbroken: How Does LLM Safety Training Fail?5 Jul

Other references · 13

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  4. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  5. 2020Scaling Laws for Neural Language Modelscited by 6,411
  6. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  7. 2022Training Compute-Optimal Large Language Modelscited by 2,927
  8. 2023A Survey of Large Language Modelscited by 2,537
  9. 2022Emergent Abilities of Large Language Modelscited by 2,483
  10. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,829
  11. 2022CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesiscited by 876
  12. 2023The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Onlycited by 755
1 more
  1. 2022Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studiescited by 506

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 99; 2025: 139; 2026: 96.

337 papers.

Sort:

By area:

  1. 2024A StrongREJECT for Empty Jailbreakscited by 352
  2. 2024SafeDecoding: Defending against Jailbreak Attacks via Safety-Aware Decodingcited by 233
  3. 2024Jailbreak Attacks and Defenses Against Large Language Models: A Surveycited by 215
  4. 2024ArtPrompt: ASCII Art-based Jailbreak Attacks against Aligned LLMscited by 197
  5. 2024Security and Privacy Challenges of Large Language Models: A Surveycited by 179
  6. 2024COLD-Attack: Jailbreaking LLMs with Stealthiness and Controllabilitycited by 162
  7. 2024Latent Adversarial Training Improves Robustness to Persistent Harmful Behaviors in LLMscited by 158
  8. 2024Agent Smith: A Single Image Can Jailbreak One Million Multimodal LLM Agents Exponentially Fastcited by 149
  9. 2024AutoDefense: Multi-Agent LLM Defense against Jailbreak Attackscited by 135
  10. 2024Attacks, Defenses and Evaluations for LLM Conversation Safety: A Surveycited by 124
  11. 2024Jailbreak Vision Language Models via Bi-Modal Adversarial Promptcited by 103
  12. 2024Weak-to-Strong Jailbreaking on Large Language Modelscited by 100
38 more
  1. 2024Improved Techniques for Optimization-Based Jailbreaking on Large Language Modelscited by 98
  2. 2024A Comprehensive Study of Jailbreak Attack versus Defense for Large Language Modelscited by 95
  3. 2025H-CoT: Hijacking the Chain-of-Thought Safety Reasoning Mechanism to Jailbreak Large Reasoning Models, Including OpenAI o1/o3, DeepSeek-R1, and Gemini 2.0 Flash Thinkingcited by 94
  4. 2024How Alignment and Jailbreak Work: Explain LLM Safety through Intermediate Hidden Statescited by 92
  5. 2024DrAttack: Prompt Decomposition and Reconstruction Makes Powerful LLM Jailbreakerscited by 91
  6. 2024Red-Teaming for Generative AI: Silver Bullet or Security Theater?cited by 81
  7. 2024Defending Large Language Models Against Jailbreak Attacks via Layer-specific Editingcited by 77
  8. 2024Safeguarding Large Language Models: A Surveycited by 73
  9. 2024FlipAttack: Jailbreak LLMs via Flippingcited by 73
  10. 2024Improved Few-Shot Jailbreaking Can Circumvent Aligned Language Models and Their Defensescited by 68
  11. 2024JailbreakZoo: Survey, Landscapes, and Horizons in Jailbreaking Large Language and Vision-Language Modelscited by 67
  12. 2024Intention Analysis Makes LLMs A Good Jailbreak Defendercited by 64
  13. 2024Speak Out of Turn: Safety Vulnerability of Large Language Models in Multi-turn Dialoguecited by 64
  14. 2024Uncovering Safety Risks of Large Language Models through Concept Activation Vectorcited by 63
  15. 2025Reasoning-Augmented Conversation for Multi-Turn Jailbreak Attacks on Large Language Modelscited by 61
  16. 2025Safety in Large Reasoning Models: A Surveycited by 58
  17. 2024Play Guessing Game with LLM: Indirect Jailbreak Attack with Implicit Cluescited by 57
  18. 2024EasyJailbreak: A Unified Framework for Jailbreaking Large Language Modelscited by 57
  19. 2025A Survey of LLM-Driven AI Agent Communication: Protocols, Security Risks, and Defense Countermeasurescited by 54
  20. 2024Jailbreak Large Vision-Language Models Through Multi-Modal Linkagecited by 53
  21. 2025Foot-In-The-Door: A Multi-turn Jailbreak for LLMscited by 50
  22. 2023Bypassing the Safety Training of Open-Source LLMs with Priming Attackscited by 48
  23. 2024Revisiting Jailbreaking for Large Language Models: A Representation Engineering Perspectivecited by 44
  24. 2024Single Image Unlearning: Efficient Machine Unlearning in Multimodal Large Language Modelscited by 44
  25. 2024Breaking Down the Defenses: A Comparative Survey of Attacks on Large Language Modelscited by 42
  26. 2024Cheating Automatic LLM Benchmarks: Null Models Achieve High Win Ratescited by 42
  27. 2024Distract Large Language Models for Automatic Jailbreak Attackcited by 40
  28. 2024Can Language Models Perform Robust Reasoning in Chain-of-thought Prompting with Noisy Rationales?cited by 40
  29. 2024Arondight: Red Teaming Large Vision Language Models with Auto-generated Multi-modal Jailbreak Promptscited by 39
  30. 2024RedAgent: Red Teaming Large Language Models with Context-aware Autonomous Language Agentcited by 39
  31. 2025A Mousetrap: Fooling Large Reasoning Models for Jailbreak with Chain of Iterative Chaoscited by 39
  32. 2025Exploring Criteria of Loss Reweighting to Enhance LLM Unlearningcited by 39
  33. 2024Boosting Jailbreak Attack with Momentumcited by 38
  34. 2024A Cross-Language Investigation into Jailbreak Attacks in Large Language Modelscited by 37
  35. 2025AdaSteer: Your Aligned LLM is Inherently an Adaptive Jailbreak Defendercited by 37
  36. 2024Emulated Disalignment: Safety Alignment for Large Language Models May Backfire!cited by 36
  37. 2025AgentSafe: Safeguarding Large Language Model-based Multi-agent Systems via Hierarchical Data Managementcited by 35
  38. 2024Alpaca against Vicuna: Using LLMs to Uncover Memorization of LLMscited by 33

50 most cited of 337 All 337 (JSON)