mlsanityData to 14 Oct 2026

One Turn Too Late: Learning When to Intervene Against Multi-Turn Malicious Intent

arXiv 2605.05630 ↗First posted 7 May 2026cs.CL cs.AI cs.CRcited by 0

Not on the map. Why?

Abstract (arXiv)

Hidden malicious intent in multi-turn dialogue poses a growing threat to deployed large language models (LLMs). Rather than exposing a harmful objective in a single prompt, attackers can distribute their intent across multiple benign-looking turns, making defense a problem not only of whether a dialogue is harmful, but also of when intervention becomes necessary. Existing trace-level labeling approaches provide only coarse safety signals and do not identify this intervention boundary, making it difficult to distinguish timely intervention from premature refusal or a block that comes too late. This work introduces turn-level harm-enabling supervision for multi-turn defense. We define the earliest harm-enabling turn as the first point at which delivering a candidate response would make the accumulated interaction sufficient to enable harmful action. To instantiate this supervision at scale, we construct the Multi-Turn Intent Dataset (MTID), which contains adaptive attack rollouts, matched benign hard negatives, and annotations of this boundary. Using MTID, we train TurnGate, a response-aware monitor that learns when to intervene, and further optimize its policy through multi-turn reinforcement learning. Experiments show that turn-level boundary supervision improves intervention localization, while reinforcement learning further improves the safety--utility trade-off. TurnGate outperforms existing guardrails and multi-turn monitoring baselines, and generalizes across risk domains, attacker pipelines, and target models. Our code is available at https://github.com/Graph-COM/TurnGate.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 38 of its 45 references that are in the map. First posted on arXiv 7 May 2026.

Builds on

38 of 45 references.

References · 38

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  3. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
  4. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,207
  5. 2024The AI Scientist: Towards Fully Automated Open-Ended Scientific Discoverycited by 1,071
  6. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Modelscited by 785
  7. 2023XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Modelscited by 505
  8. 2024Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attackcited by 390
  9. 2024Improving Alignment and Robustness with Circuit Breakerscited by 322
  10. 2023Low-Resource Languages Jailbreak GPT-4cited by 309
  11. 2024Deliberative Alignment: Reasoning Enables Safer Language Modelscited by 308
  12. 2023A Wolf in Sheep's Clothing: Generalized Nested Jailbreak Prompts can Fool Large Language Models Easilycited by 259
26 more
  1. 2024WildTeaming at Scale: From In-the-Wild Jailbreaks to (Adversarially) Safer Language Modelscited by 223
  2. 2024MDAgents: An Adaptive Collaboration of LLMs for Medical Decision-Makingcited by 163
  3. 2025Qwen3Guard Technical Reportcited by 155
  4. 2024Attacks, Defenses and Evaluations for LLM Conversation Safety: A Surveycited by 130
  5. 2025X-Teaming: Multi-Turn Jailbreaks and Defenses with Adaptive Multi-Agentscited by 125
  6. 2024DrAttack: Prompt Decomposition and Reconstruction Makes Powerful LLM Jailbreakerscited by 101
  7. 2024How Alignment and Jailbreak Work: Explain LLM Safety through Intermediate Hidden Statescited by 98
  8. 2024LLMs know their vulnerabilities: Uncover Safety Gaps through Natural Distribution Shiftscited by 86
  9. 2024Intention Analysis Makes LLMs A Good Jailbreak Defendercited by 74
  10. 2025FalseReject: A Resource for Improving Contextual Safety and Mitigating Over-Refusals in LLMs via Structured Reasoningcited by 58
  11. 2024GUARD: Role-playing to Generate Natural-language Jailbreakings to Test Guideline Adherence of Large Language Modelscited by 48
  12. 2024When LLM Meets DRL: Advancing Jailbreaking Efficiency via DRL-guided Searchcited by 47
  13. 2025LLMs in Software Security: A Survey of Vulnerability Detection Techniques and Insightscited by 47
  14. 2024Automated Red Teaming with GOAT: the Generative Offensive Agent Testercited by 45
  15. 2024RedAgent: Red Teaming Large Language Models with Context-aware Autonomous Language Agentcited by 44
  16. 2025MTSA: Multi-turn Safety Alignment for LLMs through Multi-round Red-teamingcited by 32
  17. 2025Monitoring Decomposition Attacks in LLMs with Lightweight Sequential Monitorscited by 25
  18. 2024RoleBreak: Character Hallucination as a Jailbreak Attack in Role-Playing Systemscited by 21
  19. 2025Understanding and Mitigating Overrefusal in LLMs from an Unveiling Perspective of Safety Decision Boundarycited by 18
  20. 2025Do LLMs Really Forget? Evaluating Unlearning with Knowledge Correlation and Confidence Awarenesscited by 14
  21. 2025Guardians and Offenders: A Survey on Harmful Content Generation and Safety Mitigation of LLMcited by 9
  22. 2024Ferret: Faster and Effective Automated Red Teaming with Reward-Based Scoring Techniquecited by 8
  23. 2025Benchmarking Misuse Mitigation Against Covert Adversariescited by 8
  24. 2025Prompt, Divide, and Conquer: Bypassing Large Language Model Safety Filters via Segmented and Distributed Prompt Processingcited by 7
  25. 2025The Trojan Knowledge: Bypassing Commercial LLM Guardrails via Harmless Prompt Weaving and Adaptive Tree Searchcited by 7
  26. 2026MALicious INTent Dataset and Inoculating LLMs for Enhanced Disinformation Detectioncited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025SafeTy Reasoning Elicitation Alignment for Multi-Turn Dialogues
  2. 2024Speak Out of Turn: Safety Vulnerability of Large Language Models in Multi-turn Dialogue
  3. 2026Turn-Based Structural Triggers: Structure-Conditioned Backdoors in Multi-Turn LLMs
  4. 2025MUSE: MCTS-Driven Red Teaming Framework for Enhanced Multi-Turn Dialogue Safety in Large Language Models
  5. 2026Robust Critics: Defending LLMs Against Multi-Turn Attacks
  6. 2025Tree-based Dialogue Reinforced Policy Optimization for Red-Teaming Attacks
  7. 2026The Model Plants the Trigger: Answer-Side Backdoor Attacks in Multi-Turn Large Language Models
  8. 2026THRD: A Training-Free Multi-Turn Defense Framework for Jailbreak Attacks on Large Language Models