One Turn Too Late: Learning When to Intervene Against Multi-Turn Malicious Intent
Not on the map. Why?
Abstract (arXiv)
Hidden malicious intent in multi-turn dialogue poses a growing threat to deployed large language models (LLMs). Rather than exposing a harmful objective in a single prompt, attackers can distribute their intent across multiple benign-looking turns, making defense a problem not only of whether a dialogue is harmful, but also of when intervention becomes necessary. Existing trace-level labeling approaches provide only coarse safety signals and do not identify this intervention boundary, making it difficult to distinguish timely intervention from premature refusal or a block that comes too late. This work introduces turn-level harm-enabling supervision for multi-turn defense. We define the earliest harm-enabling turn as the first point at which delivering a candidate response would make the accumulated interaction sufficient to enable harmful action. To instantiate this supervision at scale, we construct the Multi-Turn Intent Dataset (MTID), which contains adaptive attack rollouts, matched benign hard negatives, and annotations of this boundary. Using MTID, we train TurnGate, a response-aware monitor that learns when to intervene, and further optimize its policy through multi-turn reinforcement learning. Experiments show that turn-level boundary supervision improves intervention localization, while reinforcement learning further improves the safety--utility trade-off. TurnGate outperforms existing guardrails and multi-turn monitoring baselines, and generalizes across risk domains, attacker pipelines, and target models. Our code is available at https://github.com/Graph-COM/TurnGate.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 38 of its 45 references that are in the map. First posted on arXiv 7 May 2026.
Builds on
38 of 45 references.
References · 38
- 2022Training language models to follow instructions with human feedbackcited by 15,971
- 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
- 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
- 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,207
- 2024The AI Scientist: Towards Fully Automated Open-Ended Scientific Discoverycited by 1,071
- 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Modelscited by 785
- 2023XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Modelscited by 505
- 2024Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attackcited by 390
- 2024Improving Alignment and Robustness with Circuit Breakerscited by 322
- 2023Low-Resource Languages Jailbreak GPT-4cited by 309
- 2024Deliberative Alignment: Reasoning Enables Safer Language Modelscited by 308
- 2023A Wolf in Sheep's Clothing: Generalized Nested Jailbreak Prompts can Fool Large Language Models Easilycited by 259
26 more
- 2024WildTeaming at Scale: From In-the-Wild Jailbreaks to (Adversarially) Safer Language Modelscited by 223
- 2024MDAgents: An Adaptive Collaboration of LLMs for Medical Decision-Makingcited by 163
- 2025Qwen3Guard Technical Reportcited by 155
- 2024Attacks, Defenses and Evaluations for LLM Conversation Safety: A Surveycited by 130
- 2025X-Teaming: Multi-Turn Jailbreaks and Defenses with Adaptive Multi-Agentscited by 125
- 2024DrAttack: Prompt Decomposition and Reconstruction Makes Powerful LLM Jailbreakerscited by 101
- 2024How Alignment and Jailbreak Work: Explain LLM Safety through Intermediate Hidden Statescited by 98
- 2024LLMs know their vulnerabilities: Uncover Safety Gaps through Natural Distribution Shiftscited by 86
- 2024Intention Analysis Makes LLMs A Good Jailbreak Defendercited by 74
- 2025FalseReject: A Resource for Improving Contextual Safety and Mitigating Over-Refusals in LLMs via Structured Reasoningcited by 58
- 2024GUARD: Role-playing to Generate Natural-language Jailbreakings to Test Guideline Adherence of Large Language Modelscited by 48
- 2024When LLM Meets DRL: Advancing Jailbreaking Efficiency via DRL-guided Searchcited by 47
- 2025LLMs in Software Security: A Survey of Vulnerability Detection Techniques and Insightscited by 47
- 2024Automated Red Teaming with GOAT: the Generative Offensive Agent Testercited by 45
- 2024RedAgent: Red Teaming Large Language Models with Context-aware Autonomous Language Agentcited by 44
- 2025MTSA: Multi-turn Safety Alignment for LLMs through Multi-round Red-teamingcited by 32
- 2025Monitoring Decomposition Attacks in LLMs with Lightweight Sequential Monitorscited by 25
- 2024RoleBreak: Character Hallucination as a Jailbreak Attack in Role-Playing Systemscited by 21
- 2025Understanding and Mitigating Overrefusal in LLMs from an Unveiling Perspective of Safety Decision Boundarycited by 18
- 2025Do LLMs Really Forget? Evaluating Unlearning with Knowledge Correlation and Confidence Awarenesscited by 14
- 2025Guardians and Offenders: A Survey on Harmful Content Generation and Safety Mitigation of LLMcited by 9
- 2024Ferret: Faster and Effective Automated Red Teaming with Reward-Based Scoring Techniquecited by 8
- 2025Benchmarking Misuse Mitigation Against Covert Adversariescited by 8
- 2025Prompt, Divide, and Conquer: Bypassing Large Language Model Safety Filters via Segmented and Distributed Prompt Processingcited by 7
- 2025The Trojan Knowledge: Bypassing Commercial LLM Guardrails via Harmless Prompt Weaving and Adaptive Tree Searchcited by 7
- 2026MALicious INTent Dataset and Inoculating LLMs for Enhanced Disinformation Detectioncited by 3
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025SafeTy Reasoning Elicitation Alignment for Multi-Turn Dialogues
- 2024Speak Out of Turn: Safety Vulnerability of Large Language Models in Multi-turn Dialogue
- 2026Turn-Based Structural Triggers: Structure-Conditioned Backdoors in Multi-Turn LLMs
- 2025MUSE: MCTS-Driven Red Teaming Framework for Enhanced Multi-Turn Dialogue Safety in Large Language Models
- 2026Robust Critics: Defending LLMs Against Multi-Turn Attacks
- 2025Tree-based Dialogue Reinforced Policy Optimization for Red-Teaming Attacks
- 2026The Model Plants the Trigger: Answer-Side Backdoor Attacks in Multi-Turn Large Language Models
- 2026THRD: A Training-Free Multi-Turn Defense Framework for Jailbreak Attacks on Large Language Models