Automating Deception: Scalable Multi-Turn LLM Jailbreaks
On the map: LLM alignment / RLHF / preference optimisation → Safety alignment and jailbreak attacks
also in 1 more
Abstract (arXiv)
Multi-turn conversational attacks, which leverage psychological principles like Foot-in-the-Door (FITD), where a small initial request paves the way for a more significant one, to bypass safety alignments, pose a persistent threat to Large Language Models (LLMs). Progress in defending against these attacks is hindered by a reliance on manual, hard-to-scale dataset creation. This paper introduces a novel, automated pipeline for generating large-scale, psychologically-grounded multi-turn jailbreak datasets. We systematically operationalize FITD techniques into reproducible templates, creating a benchmark of 1,500 scenarios across illegal activities and offensive content. We evaluate seven models from three major LLM families under both multi-turn (with history) and single-turn (without history) conditions. Our results reveal stark differences in contextual robustness: models in the GPT family demonstrate a significant vulnerability to conversational history, with Attack Success Rates (ASR) increasing by as much as 32 percentage points. In contrast, Google's Gemini 2.5 Flash exhibits exceptional resilience, proving nearly immune to these attacks, while Anthropic's Claude 3 Haiku shows strong but imperfect resistance. These findings highlight a critical divergence in how current safety architectures handle conversational context and underscore the need for defenses that can resist narrative-based manipulation.
Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Placed in Safety alignment and jailbreak attacks, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 11 of its 22 references that are in the map. First posted on arXiv 24 Nov 2025.
Builds on
11 of 22 references.
References · 11
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
- 2024GPT-4o System Cardcited by 4,588
- 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,429
- 2024JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Modelscited by 536
- 2024Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attackcited by 390
- 2024LLM Defenses Are Not Robust to Multi-Turn Human Jailbreaks Yetcited by 151
- 2025X-Teaming: Multi-Turn Jailbreaks and Defenses with Adaptive Multi-Agentscited by 125
- 2025Foot-In-The-Door: A Multi-turn Jailbreak for LLMscited by 54
- 2025SafeDialBench: A Fine-Grained Safety Evaluation Benchmark for Large Language Models in Multi-Turn Dialogues with Diverse Jailbreak Attackscited by 23
- 2025Multi-Turn Jailbreaks Are Simpler Than They Seemcited by 17
- 2024Emerging Vulnerabilities in Frontier Models: Multi-Turn Jailbreak Attackscited by 16
Built on by
Citing papers per year: 2026: 7.
7 papers.
- 2026Not Just RLHF: Why Alignment Alone Won't Fix Multi-Agent Sycophancycited by 3
- 2026Playing Along: Learning a Double-Agent Defender for Belief Steering via Theory of Mindcited by 1
- 2026D-Judge: Disrupting Multi-Turn Jailbreaks using Semantics-Preserving Output Rewritingcited by 1
- 2026FragBench: Cross-Session Attacks Hidden in Benign-Looking Fragmentscited by 0
- 2026Scalable Hierarchical Attention Transformers for Multi-Turn Jailbreak Detection in Long Conversationscited by 0
- 2026MJ: Multi-turn LLM Jailbreaking via Decomposed Credit Assignmentcited by 0
- 2026Magnet: Detecting Cross-Session AI Misuse Through Capability Accumulationcited by 0
- 2026Magnet: Detecting Cross-Session AI Misuse Through Capability Accumulation3 Aug 2026
- 2026MJ: Multi-turn LLM Jailbreaking via Decomposed Credit Assignment13 Jul 2026
- 2026Scalable Hierarchical Attention Transformers for Multi-Turn Jailbreak Detection in Long Conversations19 Jun 2026
- 2026D-Judge: Disrupting Multi-Turn Jailbreaks using Semantics-Preserving Output Rewriting31 May 2026
- 2026Not Just RLHF: Why Alignment Alone Won't Fix Multi-Agent Sycophancy13 May 2026
- 2026FragBench: Cross-Session Attacks Hidden in Benign-Looking Fragments10 May 2026
- 2026Playing Along: Learning a Double-Agent Defender for Belief Steering via Theory of Mind13 Apr 2026
Similar, not cited
Close in wording, no citation either way
- 2025Pattern Enhanced Multi-Turn Jailbreaking: Exploiting Structural Vulnerabilities in Large Language Models
- 2026THRD: A Training-Free Multi-Turn Defense Framework for Jailbreak Attacks on Large Language Models
- 2026MultiBreak: A Scalable and Diverse Multi-turn Jailbreak Benchmark for Evaluating LLM Safety
- 2025GRAF: Multi-turn Jailbreaking via Global Refinement and Active Fabrication
- 2024Leveraging the Context through Multi-Round Interactions for Jailbreaking Attacks
- 2025Uncovering the Persuasive Fingerprint of LLMs in Jailbreaking Attacks
- 2026PsychJail: Exploring Psychological Jailbreaks via Multi-Turn Persuasion of LLM Policies
- 2024Jailbreaking and Mitigation of Vulnerabilities in Large Language Models