X-Teaming Evolutionary M2S: Automated Discovery of Multi-turn to Single-turn Jailbreak Templates
On the map: LLM safety and security → Jailbreaking
Abstract (arXiv)
Multi-turn-to-single-turn (M2S) compresses iterative red-teaming into one structured prompt, but prior work relied on a handful of manually written templates. We present X-Teaming Evolutionary M2S, an automated framework that discovers and optimizes M2S templates through language-model-guided evolution. The system pairs smart sampling from 12 sources with an LLM-as-judge inspired by StrongREJECT and records fully auditable logs. Maintaining selection pressure by setting the success threshold to θ = 0.70, we obtain five evolutionary generations, two new template families, and 44.8% overall success (103/230) on GPT-4.1. A balanced cross-model panel of 2,500 trials (judge fixed) shows that structural gains transfer but vary by target; two models score zero at the same threshold. We also find a positive coupling between prompt length and score, motivating length-aware judging. Our results demonstrate that structure-level search is a reproducible route to stronger single-turn probes and underscore the importance of threshold calibration and cross-model evaluation. Code, configurations, and artifacts are available at https://github.com/hyunjun1121/M2S-x-teaming.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Jailbreaking, one of 12 directions in LLM safety and security. Builds on 17 of its 21 references that are in the map. First posted on arXiv 10 Sep 2025.
Builds on
17 of 21 references.
Closest ancestors · 7
Same direction, newest first
- 2025X-Teaming: Multi-Turn Jailbreaks and Defenses with Adaptive Multi-Agents15 Apr
- 2025M2S: Multi-turn to Single-turn jailbreak in Red Teaming for LLMs6 Mar
- 2024LLM Defenses Are Not Robust to Multi-Turn Human Jailbreaks Yet27 Aug
- 2024A StrongREJECT for Empty Jailbreaks15 Feb
- 2023Scalable and Transferable Black-Box Jailbreaks for Language Models via Persona Modulation6 Nov
- 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul
- 2023MasterKey: Automated Jailbreak Across Multiple Large Language Model Chatbots16 Jul
Other references · 10
- 2022Training language models to follow instructions with human feedbackcited by 15,971
- 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
- 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
- 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,429
- 2022Improving alignment of dialogue agents via targeted human judgementscited by 531
- 2025Humanity's Last Examcited by 491
- 2024Improving Alignment and Robustness with Circuit Breakerscited by 322
- 2023Pretraining Language Models with Human Preferencescited by 247
- 2023Removing RLHF Protections in GPT-4 via Fine-Tuningcited by 171
- 2025ObjexMT: Objective Extraction and Metacognitive Calibration for LLM-as-a-Judge under Multi-Turn Jailbreakscited by 3
Built on by
Citing papers per year: 2025: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2025Evolve the Method, Not the Prompts: Evolutionary Synthesis of Jailbreak Attacks on LLMs
- 2026EvoJail: Evolutionary Diverse Jailbreak Prompt Generation for Large Language Models
- 2025Beyond Fixed and Dynamic Prompts: Embedded Jailbreak Templates for Advancing LLM Security
- 2026JailbreakOPT: Tool-Assisted Iterative Jailbreak Prompt Optimization
- 2026AJAR: Adaptive Jailbreak Architecture for Red-teaming
- 2026Reasoning as an Attack Surface: Adaptive Evolutionary CoT Jailbreaks for LLMs
- 2026Red-TTT: Test-Time Training for Automated Jailbreaking Large Language Models
- 2025Align to Misalign: Automatic LLM Jailbreak with Meta-Optimized LLM Judges