# Tempest: Autonomous Multi-Turn Jailbreaking of Large Language Models with Tree Search

arXiv: https://arxiv.org/abs/2503.10619 · HTML: https://mlsanity.com/paper/2503.10619/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Placed in Jailbreaking, one of 12 directions in LLM safety and security.
- Builds on 13 of its 21 references that are in the map.
- First posted on arXiv 13 Mar 2025.

## Placed in

- LLM safety and security / [Jailbreaking](https://mlsanity.com/safety/jailbreaking/)

## Builds on (most cited first, 13 of 13)

- [Universal and Transferable Adversarial Attacks on Aligned Language Models](https://mlsanity.com/paper/2307.15043/) (2023)
- [HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal](https://mlsanity.com/paper/2402.04249/) (2024)
- [Jailbreaking Black Box Large Language Models in Twenty Queries](https://mlsanity.com/paper/2310.08419/) (2023)
- [Tree of Attacks: Jailbreaking Black-Box LLMs Automatically](https://mlsanity.com/paper/2312.02119/) (2023)
- [How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMs](https://mlsanity.com/paper/2401.06373/) (2024)
- [Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attack](https://mlsanity.com/paper/2404.01833/) (2024)
- [AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts](https://mlsanity.com/paper/2010.15980/) (2020)
- [Gradient-based Adversarial Attacks against Text Transformers](https://mlsanity.com/paper/2104.13733/) (2021)
- [WildTeaming at Scale: From In-the-Wild Jailbreaks to (Adversarially) Safer Language Models](https://mlsanity.com/paper/2406.18510/) (2024)
- [Scalable and Transferable Black-Box Jailbreaks for Language Models via Persona Modulation](https://mlsanity.com/paper/2311.03348/) (2023)
- [LLM Defenses Are Not Robust to Multi-Turn Human Jailbreaks Yet](https://mlsanity.com/paper/2408.15221/) (2024)
- [LLMs know their vulnerabilities: Uncover Safety Gaps through Natural Distribution Shifts](https://mlsanity.com/paper/2410.10700/) (2024)
- [Automated Red Teaming with GOAT: the Generative Offensive Agent Tester](https://mlsanity.com/paper/2410.01606/) (2024)

## Built on by (most cited first, 17 of 17)

- [A Representation Engineering Perspective on the Effectiveness of Multi-Turn Jailbreaks](https://mlsanity.com/paper/2507.02956/) (2025)
- [Can AI agents conduct open-ended AI research? Early evidence from two case studies](https://mlsanity.com/paper/2607.27191/) (2026)
- [AutoAdv: Automated Adversarial Prompting for Multi-Turn Jailbreaking of Large Language Models](https://mlsanity.com/paper/2507.01020/) (2025)
- [The Trojan Knowledge: Bypassing Commercial LLM Guardrails via Harmless Prompt Weaving and Adaptive Tree Search](https://mlsanity.com/paper/2512.01353/) (2025)
- [Automatic LLM Red Teaming](https://mlsanity.com/paper/2508.04451/) (2025)
- [SciIntegrity-Bench: A Benchmark for Evaluating Academic Integrity in AI Scientist Systems](https://mlsanity.com/paper/2605.10246/) (2026)
- [SoK: Robustness in Large Language Models against Jailbreak Attacks](https://mlsanity.com/paper/2605.05058/) (2026)
- [Not All Turns Matter: Credit Assignment for Multi-Turn Jailbreaking](https://mlsanity.com/paper/2605.08778/) (2026)
- [CoopGuard: Stateful Cooperative Agents Safeguarding LLMs Against Evolving Multi-Round Attacks](https://mlsanity.com/paper/2604.04060/) (2026)
- [Multi-Turn Adaptive Prompting Attack on Large Vision-Language Models](https://mlsanity.com/paper/2602.14399/) (2026)
- [The Salami Slicing Threat: Exploiting Cumulative Risks in LLM Systems](https://mlsanity.com/paper/2604.11309/) (2026)
- [SoK: Intent-Oriented Systematization of Multi-Turn LLM Jailbreaks](https://mlsanity.com/paper/2608.01117/) (2026)
- [AutoAdv: Automated Adversarial Prompting for Multi-Turn Jailbreaking of Large Language Models](https://mlsanity.com/paper/2511.02376/) (2025)
- [Jailbreaking the Matrix: Nullspace Steering for Controlled Model Subversion](https://mlsanity.com/paper/2604.10326/) (2026)
- [Forecasting Trajectory-Level Safety Risks in Black-Box Multi-Turn Interactions](https://mlsanity.com/paper/2607.26820/) (2026)
- [Stateful Cooperative Agents Safeguarding LLMs Against Evolving Multi-Turn Attacks](https://mlsanity.com/paper/2608.00134/) (2026)
- [Position: Want Better ML Reviews? Stop Asking Nicely and Start Incentivizing with a Credit System](https://mlsanity.com/paper/2608.14571/) (2026)

## Abstract (arXiv)

We introduce Tempest, a multi-turn adversarial framework that models the gradual erosion of Large Language Model (LLM) safety through a tree search perspective. Unlike single-turn jailbreaks that rely on one meticulously engineered prompt, Tempest expands the conversation at each turn in a breadth-first fashion, branching out multiple adversarial prompts that exploit partial compliance from previous responses. By tracking these incremental policy leaks and re-injecting them into subsequent queries, Tempest reveals how minor concessions can accumulate into fully disallowed outputs. Evaluations on the JailbreakBench dataset show that Tempest achieves a 100% success rate on GPT-3.5-turbo and 97% on GPT-4 in a single multi-turn run, using fewer queries than baselines such as Crescendo or GOAT. This tree search methodology offers an in-depth view of how model safeguards degrade over successive dialogue turns, underscoring the urgency of robust multi-turn testing procedures for language models.