mlsanityData to 7 Oct 2026

AMT-X: Phase-Structured Multi-Turn Red-Teaming with Checklist-Gated Evaluation

arXiv 2607.11151 ↗First posted 13 Jul 2026cs.CR cs.AIcited by 0

On the map: LLM safety and security → Prompt injection and agent security

Abstract (arXiv)

Safety evaluation of large language models (LLMs) relies largely on single-turn attack datasets and single-judge scoring, underestimating risk from adaptive multi-turn adversaries and reporting a single success rate that does not separate partially actionable outputs from those carrying complete operational detail. We propose AMT-X (Adaptive Multi-Turn Exploitation), a phase-structured multi-turn red-teaming framework. Unlike prior multi-turn attacks that rely on ad hoc escalation or free-form per-goal plans, AMT-X casts the attack as an explicit, reproducible multi-phase state machine driven by semantic signals from the victim, and replaces single-judge scoring with a multi-role jury whose phase-conditioned checklists gate success on actionable harm. Across six frontier victim models (queried under their default safety alignment, without added moderation layers) and seven Moderation sub-categories, AMT-X attains overall attack success rates of 97.6-100% under a lenient score threshold, but 66.7-78.6% under a stricter gate requiring complete, real, and operational detail: a gap of up to 33 percentage points between partially and fully actionable harm.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Prompt injection and agent security, one of 12 directions in LLM safety and security. Builds on 26 of its 39 references that are in the map. First posted on arXiv 13 Jul 2026.

Builds on

26 of 39 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024Automated Red Teaming with GOAT: the Generative Offensive Agent Tester2 Oct
  2. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal6 Feb
  3. 2022Red Teaming Language Models with Language Models7 Feb

Other references · 23

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  4. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  5. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  6. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
  7. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,326
  8. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,106
  9. 2024LLM Evaluators Recognize and Favor Their Own Generationscited by 651
  10. 2023Tree of Attacks: Jailbreaking Black-Box LLMs Automaticallycited by 619
  11. 2024From Crowdsourced Data to High-Quality Benchmarks: Arena-Hard and BenchBuilder Pipelinecited by 484
  12. 2023Jailbreaking ChatGPT via Prompt Engineering: An Empirical Studycited by 478
11 more
  1. 2023Large Language Models Are Not Robust Multiple Choice Selectorscited by 454
  2. 2023Detecting Language Model Attacks with Perplexitycited by 411
  3. 2024Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attackcited by 364
  4. 2024KTO: Model Alignment as Prospect Theoretic Optimizationcited by 256
  5. 2023A Wolf in Sheep's Clothing: Generalized Nested Jailbreak Prompts can Fool Large Language Models Easilycited by 242
  6. 2024COLD-Attack: Jailbreaking LLMs with Stealthiness and Controllabilitycited by 162
  7. 2023Sociotechnical Safety Evaluation of Generative AI Systemscited by 154
  8. 2023Benchmarking Cognitive Biases in Large Language Models as Evaluatorscited by 147
  9. 2024LLM Defenses Are Not Robust to Multi-Turn Human Jailbreaks Yetcited by 145
  10. 2025X-Teaming: Multi-Turn Jailbreaks and Defenses with Adaptive Multi-Agentscited by 112
  11. 2024DrAttack: Prompt Decomposition and Reconstruction Makes Powerful LLM Jailbreakerscited by 91

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026OpenRT: An Open-Source Red Teaming Framework for Multimodal LLMs
  2. 2024Code-Switching Red-Teaming: LLM Evaluation for Safety and Multilingual Understanding
  3. 2026NRT-Bench: Benchmarking Multi-Turn Red-Teaming of LLM Operator Agents in Safety-Critical Control Rooms
  4. 2025MUSE: MCTS-Driven Red Teaming Framework for Enhanced Multi-Turn Dialogue Safety in Large Language Models
  5. 2025MTSA: Multi-turn Safety Alignment for LLMs through Multi-round Red-teaming
  6. 2025LLM-Safety Evaluations Lack Robustness
  7. 2026AutoRISE: Agent-Driven Strategy Evolution for Red-Teaming Large Language Models
  8. 2025MCP-SafetyBench: A Benchmark for Safety Evaluation of Large Language Models with Real-World MCP Servers