mlsanityData to 14 Oct 2026

Dialogue Injection Attack: Jailbreaking LLMs through Context Manipulation

arXiv 2503.08195 ↗First posted 11 Mar 2025cs.CLcited by 14

On the map: LLM safety and security → Jailbreaking

Abstract (arXiv)

Large language models (LLMs) have demonstrated significant utility in a wide range of applications; however, their deployment is plagued by security vulnerabilities, notably jailbreak attacks. These attacks manipulate LLMs to generate harmful or unethical content by crafting adversarial prompts. While much of the current research on jailbreak attacks has focused on single-turn interactions, it has largely overlooked the impact of historical dialogues on model behavior. In this paper, we introduce a novel jailbreak paradigm, Dialogue Injection Attack (DIA), which leverages the dialogue history to enhance the success rates of such attacks. DIA operates in a black-box setting, requiring only access to the chat API or knowledge of the LLM's chat template. We propose two methods for constructing adversarial historical dialogues: one adapts gray-box prefilling attacks, and the other exploits deferred responses. Our experiments show that DIA achieves state-of-the-art attack success rates on recent LLMs, including Llama-3.1 and GPT-4o. Additionally, we demonstrate that DIA can bypass 5 different defense mechanisms, highlighting its robustness and effectiveness.

Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Placed in Jailbreaking, one of 12 directions in LLM safety and security. Builds on 62 of its 93 references that are in the map. First posted on arXiv 11 Mar 2025.

Builds on

62 of 93 references.

Closest ancestors · 22

Same direction, newest first

  1. 2024AdaPPA: Adaptive Position Pre-Fill Jailbreak Attack Approach Targeting LLMs11 Sep
  2. 2024ChatBug: A Common Vulnerability of Aligned LLMs Induced by Chat Templates17 Jun
  3. 2024Defensive Prompt Patch: A Robust and Interpretable Defense of LLMs against Jailbreak Attacks30 May
  4. 2024Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attacks2 Apr
  5. 2024Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attack2 Apr
  6. 2024JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Models28 Mar
  7. 2024Don't Listen To Me: Understanding and Exploring Jailbreak Prompts of Large Language Models26 Mar
  8. 2024AutoDefense: Multi-Agent LLM Defense against Jailbreak Attacks2 Mar
  9. 2024Making Them Ask and Answer: Jailbreaking Large Language Models in Few Queries via Disguise and Reconstruction28 Feb
  10. 2024A Comprehensive Study of Jailbreak Attack versus Defense for Large Language Models21 Feb
  11. 2024Play Guessing Game with LLM: Indirect Jailbreak Attack with Implicit Clues14 Feb
  12. 2024Intention Analysis Makes LLMs A Good Jailbreak Defender12 Jan
10 more
  1. 2023A Wolf in Sheep's Clothing: Generalized Nested Jailbreak Prompts can Fool Large Language Models Easily14 Nov
  2. 2023DeepInception: Hypnotize Large Language Model to Be Jailbreaker6 Nov
  3. 2023AutoDAN: Interpretable Gradient-Based Adversarial Attacks on Large Language Models23 Oct
  4. 2023Jailbreaking Black Box Large Language Models in Twenty Queries12 Oct
  5. 2023Catastrophic Jailbreak of Open-source LLMs via Exploiting Generation10 Oct
  6. 2023Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrations10 Oct
  7. 2023Defending Against Alignment-Breaking Attacks via Robustly Aligned LLM18 Sep
  8. 2023"Do Anything Now": Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Models7 Aug
  9. 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul
  10. 2023MasterKey: Automated Jailbreak Across Multiple Large Language Model Chatbots16 Jul

Other references · 40

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2024The Llama 3 Herd of Modelscited by 14,715
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  5. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  6. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  7. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  8. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  9. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  10. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,998
  11. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,434
  12. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,154
28 more
  1. 2024Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluatorscited by 897
  2. 2023BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Datasetcited by 860
  3. 2023H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Modelscited by 832
  4. 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Modelscited by 651
  5. 2023YaRN: Efficient Context Window Extension of Large Language Modelscited by 601
  6. 2024ORPO: Monolithic Preference Optimization without Reference Modelcited by 562
  7. 2024Safety Alignment Should Be Made More Than Just a Few Tokens Deepcited by 472
  8. 2023Ring Attention with Blockwise Transformers for Near-Infinite Contextcited by 457
  9. 2023Multi-step Jailbreaking Privacy Attacks on ChatGPTcited by 333
  10. 2023Effective Long-Context Scaling of Foundation Modelscited by 303
  11. 2024InfiniGen: Efficient Generative Inference of Large Language Models with Dynamic KV Cache Managementcited by 245
  12. 2023LLM Self Defense: By Self Examination, LLMs Know They Are Being Trickedcited by 244
  13. 2024ShieldGemma: Generative AI Content Moderation Based on Gemmacited by 243
  14. 2024Mooncake: A KVCache-centric Disaggregated Architecture for LLM Servingcited by 196
  15. 2023MART: Improving LLM Safety with Multi-round Automatic Red-Teamingcited by 163
  16. 2024On Prompt-Driven Safeguarding for Large Language Modelscited by 116
  17. 2024Pride and Prejudice: LLM Amplifies Self-Bias in Self-Refinementcited by 114
  18. 2024Data-efficient Fine-tuning for LLM-based Recommendationcited by 104
  19. 2024RazorAttention: Efficient KV Cache Compression Through Retrieval Headscited by 100
  20. 2024Bias and Unfairness in Information Retrieval Systems: New Challenges in the LLM Eracited by 98
  21. 2023Identifying and Mitigating the Security Risks of Generative AIcited by 91
  22. 2024Long Is More for Alignment: A Simple but Tough-to-Beat Baseline for Instruction Fine-Tuningcited by 85
  23. 2024Does Refusal Training in LLMs Generalize to the Past Tense?cited by 79
  24. 2024Learning or Self-aligning? Rethinking Instruction Fine-tuningcited by 43
  25. 2024TransLLaMa: LLM-based Simultaneous Translation Systemcited by 34
  26. 2024Malla: Demystifying Real-world Large Language Model Integrated Malicious Servicescited by 33
  27. 2023Bergeron: Combating Adversarial Attacks through a Conscience-Based Alignment Frameworkcited by 30
  28. 2023Safety Alignment in NLP Tasks: Weakly Aligned Summarization as an In-Context Attackcited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 6; 2026: 8.

14 papers.

Sort:

By area:

  1. 2025Multilingual and Multi-Accent Jailbreaking of Audio LLMscited by 31
  2. 2025Visual Contextual Attack: Jailbreaking MLLMs with Image-Driven Context Injectioncited by 25
  3. 2026Prompt Injection as Role Confusioncited by 17
  4. 2025Where to Start Alignment? Diffusion Large Language Model May Demand a Distinct Positioncited by 11
  5. 2025Web Intellectual Property at Risk: Preventing Unauthorized Real-Time Retrieval by Large Language Modelscited by 3
  6. 2026JailMeter: An Evidence-Based Evaluation Framework for Jailbreak Attacks on Large Language Modelscited by 2
  7. 2025PD3F: A Pluggable and Dynamic DoS-Defense Framework Against Resource Consumption Attacks Targeting Large Language Modelscited by 1
  8. 2026SaFeR-Steer: Evolving Multi-Turn MLLMs via Synthetic Bootstrapping and Feedback Dynamicscited by 1
  9. 2026Benign Fine-Tuning Breaks Safety Alignment in Audio LLMscited by 1
  10. 2026DataShield: Safety-degrading Data Filtering for LLM Benign Instruction Fine-Tuningcited by 1
  11. 2025The Emotional Baby Is Truly Deadly: Does your Multimodal Large Reasoning Model Have Emotional Flattery towards Humans?cited by 0
  12. 2026Persona Attack: Incremental Memory Injection Jailbreak Attack against Large Language Modelscited by 0
2 more
  1. 2026Stress Testing Unlearning Algorithmscited by 0
  2. 2026DuplexJail: Safety Alignment Breaks Under Spoken Interruption in Full-Duplex Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Jailbreaking and Mitigation of Vulnerabilities in Large Language Models
  2. 2024Jailbreak Attacks and Defenses Against Large Language Models: A Survey
  3. 2025AutoAdv: Automated Adversarial Prompting for Multi-Turn Jailbreaking of Large Language Models
  4. 2024Defending Large Language Models Against Jailbreak Attacks via Layer-specific Editing
  5. 2025AutoAdv: Automated Adversarial Prompting for Multi-Turn Jailbreaking of Large Language Models
  6. 2025ChatInject: Abusing Chat Templates for Prompt Injection in LLM Agents
  7. 2025Automating Deception: Scalable Multi-Turn LLM Jailbreaks
  8. 2024JailbreakHunter: A Visual Analytics Approach for Jailbreak Prompts Discovery from Large-Scale Human-LLM Conversational Datasets