mlsanityData to 7 Oct 2026

Leveraging the Context through Multi-Round Interactions for Jailbreaking Attacks

arXiv 2402.09177 ↗First posted 14 Feb 2024cs.LG cs.AI cs.CLcited by 32

On the map: LLM safety and security → Jailbreaking

Abstract (arXiv)

Large Language Models (LLMs) are susceptible to Jailbreaking attacks, which aim to extract harmful information by subtly modifying the attack query. As defense mechanisms evolve, directly obtaining harmful information becomes increasingly challenging for Jailbreaking attacks. In this work, inspired from Chomsky's transformational-generative grammar theory and human practices of indirect context to elicit harmful information, we focus on a new attack form, called Contextual Interaction Attack. We contend that the prior contextu2014the information preceding the attack queryu2014plays a pivotal role in enabling strong Jailbreaking attacks. Specifically, we propose a first multi-turn approach that leverages benign preliminary questions to interact with the LLM. Due to the autoregressive nature of LLMs, which use previous conversation rounds as context during generation, we guide the model's question-response pair to construct a context that is semantically aligned with the attack query to execute the attack. We conduct experiments on seven different LLMs and demonstrate the efficacy of this attack, which is black-box and can also transfer across LLMs. We believe this can lead to further developments and understanding of security in LLMs.

Key facts. Cited by 32 papers in the map (data through 7 Oct 2026). Placed in Jailbreaking, one of 12 directions in LLM safety and security. Builds on 39 of its 63 references that are in the map. First posted on arXiv 14 Feb 2024.

Builds on

39 of 63 references.

Closest ancestors · 12

Same direction, newest first

  1. 2023Analyzing the Inherent Response Tendency of LLMs: Real-World Instructions-Driven Jailbreak7 Dec
  2. 2023Tree of Attacks: Jailbreaking Black-Box LLMs Automatically4 Dec
  3. 2023Cognitive Overload: Jailbreaking Large Language Models with Overloaded Logical Thinking16 Nov
  4. 2023Jailbreaking Black Box Large Language Models in Twenty Queries12 Oct
  5. 2023Catastrophic Jailbreak of Open-source LLMs via Exploiting Generation10 Oct
  6. 2023Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrations10 Oct
  7. 2023SmoothLLM: Defending Large Language Models Against Jailbreaking Attacks5 Oct
  8. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models3 Oct
  9. 2023GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Prompts19 Sep
  10. 2023GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Cipher12 Aug
  11. 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul
  12. 2023Jailbroken: How Does LLM Safety Training Fail?5 Jul

Other references · 27

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2023GPT-4 Technical Reportcited by 18,132
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  5. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  6. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  7. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  8. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  9. 2023Mistral 7Bcited by 3,065
  10. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,801
  11. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,116
  12. 2023Extending Context Window of Large Language Models via Positional Interpolationcited by 721
15 more
  1. 2022Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learnedcited by 702
  2. 2022What Can Transformers Learn In-Context? A Case Study of Simple Function Classescited by 700
  3. 2019Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailmentcited by 663
  4. 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Modelscited by 608
  5. 2023Exploiting Programmatic Behavior of LLMs: Dual-Use Through Standard Security Attackscited by 268
  6. 2018Adversarial Attacks and Defences: A Surveycited by 259
  7. 2023Red-Teaming Large Language Models using Chain of Utterances for Safety-Alignmentcited by 213
  8. 2022Teaching Algorithmic Reasoning via In-context Learningcited by 111
  9. 2023Attention Sorting Combats Recency Bias In Long Context Language Modelscited by 89
  10. 2023Adversarial Demonstration Attacks on Large Language Modelscited by 58
  11. 2021Defining and Detecting Toxicity on Social Media: Context and Knowledge are Keycited by 29
  12. 2023What can we learn from Data Leakage and Unlearning for Law?cited by 15
  13. 2016Efficient Certified RAT Verificationcited by 12
  14. 2022Assessing the impact of contextual information in hate speech detectioncited by 10
  15. 2022Debiased Large Language Models Still Associate Muslims with Uniquely Violent Actscited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 8; 2025: 16; 2026: 8.

32 papers.

Sort:

By area:

  1. 2024Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attackscited by 450
  2. 2024Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attackcited by 364
  3. 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 110
  4. 2025A Survey on Trustworthy LLM Agents: Threats and Countermeasurescited by 93
  5. 2024Navigating the Risks: A Survey of Security, Privacy, and Ethics Threats in LLM-Based Agentscited by 63
  6. 2024Refusal-Trained LLMs Are Easily Jailbroken As Browser Agentscited by 61
  7. 2025Reasoning-Augmented Conversation for Multi-Turn Jailbreak Attacks on Large Language Modelscited by 61
  8. 2024How (un)ethical are instruction-centric responses of LLMs? Unveiling the vulnerabilities of safety guardrails to harmful queriescited by 26
  9. 2024You Know What I'm Saying: Jailbreak Attack via Implicit Referencecited by 13
  10. 2025Multi-lingual Multi-turn Automated Red Teaming for LLMscited by 12
  11. 2024Diversity Helps Jailbreak Large Language Modelscited by 9
  12. 2025AutoAdv: Automated Adversarial Prompting for Multi-Turn Jailbreaking of Large Language Modelscited by 8
20 more
  1. 2025AutoRAN: Automated Hijacking of Safety Reasoning in Large Reasoning Modelscited by 7
  2. 2026GRP-Obliteration: Unaligning LLMs With a Single Unlabeled Promptcited by 6
  3. 2025On the Robustness of Transformers against Context Hijacking for Linear Classificationcited by 4
  4. 2025REVEAL: Multi-turn Evaluation of Image-Input Harms for Vision LLMcited by 4
  5. 2025SoK: Systematizing LLM Prompt Security: Taxonomies, Datasets, and Unified Evaluation of Attacks and Defensescited by 4
  6. 2024Preventing Jailbreak Prompts as Malicious Tools for Cybercriminals: A Cyber Defense Perspectivecited by 3
  7. 2025VRSA: Jailbreaking Multimodal Large Language Models through Visual Reasoning Sequential Attackcited by 2
  8. 2025Linearly Decoding Refused Knowledge in Aligned Language Modelscited by 1
  9. 2025Layer-Wise Perturbations via Sparse Autoencoders for Adversarial Text Generationcited by 1
  10. 2026Improving Safety Alignment via Balanced Direct Preference Optimizationcited by 1
  11. 2026Prompt Attack Detection with LLM-as-a-Judge and Mixture-of-Modelscited by 1
  12. 2026The Salami Slicing Threat: Exploiting Cumulative Risks in LLM Systemscited by 1
  13. 2026One Word at a Time: Incremental Completion Decomposition Breaks LLM Safetycited by 1
  14. 2025GRAF: Multi-turn Jailbreaking via Global Refinement and Active Fabricationcited by 0
  15. 2025A Conceptual Framework for AI Capability Evaluationscited by 0
  16. 2025RedTopic: Toward Topic-Diverse Red Teaming of Large Language Modelscited by 0
  17. 2025AutoAdv: Automated Adversarial Prompting for Multi-Turn Jailbreaking of Large Language Modelscited by 0
  18. 2026Robust Critics: Defending LLMs Against Multi-Turn Attackscited by 0
  19. 2026Execution-Grounded Security Testing for Coding Agents in Software Engineering Pipelinescited by 0
  20. 2026You Are What You Read: Misalignment via In-Context Persona Inductioncited by 0