mlsanityData to 14 Oct 2026

Tempest: Autonomous Multi-Turn Jailbreaking of Large Language Models with Tree Search

arXiv 2503.10619 ↗First posted 13 Mar 2025cs.AI cs.CL cs.CRcited by 17

On the map: LLM safety and security → Jailbreaking

Abstract (arXiv)

We introduce Tempest, a multi-turn adversarial framework that models the gradual erosion of Large Language Model (LLM) safety through a tree search perspective. Unlike single-turn jailbreaks that rely on one meticulously engineered prompt, Tempest expands the conversation at each turn in a breadth-first fashion, branching out multiple adversarial prompts that exploit partial compliance from previous responses. By tracking these incremental policy leaks and re-injecting them into subsequent queries, Tempest reveals how minor concessions can accumulate into fully disallowed outputs. Evaluations on the JailbreakBench dataset show that Tempest achieves a 100% success rate on GPT-3.5-turbo and 97% on GPT-4 in a single multi-turn run, using fewer queries than baselines such as Crescendo or GOAT. This tree search methodology offers an in-depth view of how model safeguards degrade over successive dialogue turns, underscoring the urgency of robust multi-turn testing procedures for language models.

Key facts. Cited by 17 papers in the map (data through 14 Oct 2026). Placed in Jailbreaking, one of 12 directions in LLM safety and security. Builds on 13 of its 21 references that are in the map. First posted on arXiv 13 Mar 2025.

Builds on

13 of 21 references.

Closest ancestors · 8

Same direction, newest first

  1. 2024LLM Defenses Are Not Robust to Multi-Turn Human Jailbreaks Yet27 Aug
  2. 2024WildTeaming at Scale: From In-the-Wild Jailbreaks to (Adversarially) Safer Language Models26 Jun
  3. 2024Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attack2 Apr
  4. 2024How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMs12 Jan
  5. 2023Tree of Attacks: Jailbreaking Black-Box LLMs Automatically4 Dec
  6. 2023Scalable and Transferable Black-Box Jailbreaks for Language Models via Persona Modulation6 Nov
  7. 2023Jailbreaking Black Box Large Language Models in Twenty Queries12 Oct
  8. 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul

Other references · 5

  1. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,429
  2. 2020AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Promptscited by 249
  3. 2021Gradient-based Adversarial Attacks against Text Transformerscited by 234
  4. 2024LLMs know their vulnerabilities: Uncover Safety Gaps through Natural Distribution Shiftscited by 86
  5. 2024Automated Red Teaming with GOAT: the Generative Offensive Agent Testercited by 45

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 5; 2026: 12.

17 papers.

Sort:

By area:

  1. 2025A Representation Engineering Perspective on the Effectiveness of Multi-Turn Jailbreakscited by 10
  2. 2026Can AI agents conduct open-ended AI research? Early evidence from two case studiescited by 10
  3. 2025AutoAdv: Automated Adversarial Prompting for Multi-Turn Jailbreaking of Large Language Modelscited by 8
  4. 2025The Trojan Knowledge: Bypassing Commercial LLM Guardrails via Harmless Prompt Weaving and Adaptive Tree Searchcited by 7
  5. 2025Automatic LLM Red Teamingcited by 5
  6. 2026SciIntegrity-Bench: A Benchmark for Evaluating Academic Integrity in AI Scientist Systemscited by 5
  7. 2026SoK: Robustness in Large Language Models against Jailbreak Attackscited by 3
  8. 2026Not All Turns Matter: Credit Assignment for Multi-Turn Jailbreakingcited by 3
  9. 2026CoopGuard: Stateful Cooperative Agents Safeguarding LLMs Against Evolving Multi-Round Attackscited by 2
  10. 2026Multi-Turn Adaptive Prompting Attack on Large Vision-Language Modelscited by 1
  11. 2026The Salami Slicing Threat: Exploiting Cumulative Risks in LLM Systemscited by 1
  12. 2026SoK: Intent-Oriented Systematization of Multi-Turn LLM Jailbreakscited by 1
5 more
  1. 2025AutoAdv: Automated Adversarial Prompting for Multi-Turn Jailbreaking of Large Language Modelscited by 0
  2. 2026Jailbreaking the Matrix: Nullspace Steering for Controlled Model Subversioncited by 0
  3. 2026Forecasting Trajectory-Level Safety Risks in Black-Box Multi-Turn Interactionscited by 0
  4. 2026Stateful Cooperative Agents Safeguarding LLMs Against Evolving Multi-Turn Attackscited by 0
  5. 2026Position: Want Better ML Reviews? Stop Asking Nicely and Start Incentivizing with a Credit Systemcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026MultiBreak: A Scalable and Diverse Multi-turn Jailbreak Benchmark for Evaluating LLM Safety
  2. 2026Adversarial Reframing: A Framework for Targeted Generation in Language Models
  3. 2026Transient Turn Injection: Exposing Stateless Multi-Turn Vulnerabilities in Large Language Models
  4. 2025M2S: Multi-turn to Single-turn jailbreak in Red Teaming for LLMs
  5. 2025GRAF: Multi-turn Jailbreaking via Global Refinement and Active Fabrication
  6. 2025LatentBreak: Jailbreaking Large Language Models through Latent Space Feedback
  7. 2025ForgeDAN: An Evolutionary Framework for Jailbreaking Aligned Large Language Models
  8. 2024Defending Large Language Models Against Jailbreak Attacks via Layer-specific Editing