# How Many Iterations to Jailbreak? Dynamic Budget Allocation for Multi-Turn LLM Evaluation

arXiv: https://arxiv.org/abs/2605.06605 · HTML: https://mlsanity.com/paper/2605.06605/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Jailbreaking, one of 12 directions in LLM safety and security.
- Builds on 23 of its 77 references that are in the map.
- First posted on arXiv 7 May 2026.

## Placed in

- LLM safety and security / [Jailbreaking](https://mlsanity.com/safety/jailbreaking/)

## Builds on (most cited first, 23 of 23)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2204.05862/) (2022)
- [Qwen Technical Report](https://mlsanity.com/paper/2309.16609/) (2023)
- [Universal and Transferable Adversarial Attacks on Aligned Language Models](https://mlsanity.com/paper/2307.15043/) (2023)
- [HuggingFace's Transformers: State-of-the-art Natural Language Processing](https://mlsanity.com/paper/1910.03771/) (2019)
- [Gemma 3 Technical Report](https://mlsanity.com/paper/2503.19786/) (2025)
- [Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations](https://mlsanity.com/paper/2312.06674/) (2023)
- [Red Teaming Language Models with Language Models](https://mlsanity.com/paper/2202.03286/) (2022)
- [Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned](https://mlsanity.com/paper/2209.07858/) (2022)
- [Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAs](https://mlsanity.com/paper/2503.01743/) (2025)
- [SmoothLLM: Defending Large Language Models Against Jailbreaking Attacks](https://mlsanity.com/paper/2310.03684/) (2023)
- [Multilingual E5 Text Embeddings: A Technical Report](https://mlsanity.com/paper/2402.05672/) (2024)
- [Certifying LLM Safety against Adversarial Prompting](https://mlsanity.com/paper/2309.02705/) (2023)
- [SimpleSafetyTests: a Test Suite for Identifying Critical Safety Risks in Large Language Models](https://mlsanity.com/paper/2311.08370/) (2023)
- [Automating Customer Service using LangChain: Building custom open-source GPT Chatbot for organizations](https://mlsanity.com/paper/2310.05421/) (2023)
- [PAC Confidence Sets for Deep Neural Networks via Calibrated Prediction](https://mlsanity.com/paper/2001.00106/) (2019)
- [Constitutional Classifiers++: Efficient Production-Grade Defenses against Universal Jailbreaks](https://mlsanity.com/paper/2601.04603/) (2026)
- [Active Evaluation Acquisition for Efficient LLM Benchmarking](https://mlsanity.com/paper/2410.05952/) (2024)
- [Efficient Evaluation of LLM Performance with Statistical Guarantees](https://mlsanity.com/paper/2601.20251/) (2026)
- [A Conformal Prediction Score that is Robust to Label Noise](https://mlsanity.com/paper/2405.02648/) (2024)
- [Cer-Eval: Certifiable and Cost-Efficient Evaluation Framework for LLMs](https://mlsanity.com/paper/2505.03814/) (2025)
- [Doubly Robust Conformalized Survival Analysis with Right-Censored Data](https://mlsanity.com/paper/2412.09729/) (2024)
- [Conformal Survival Bands for Risk Screening under Right-Censoring](https://mlsanity.com/paper/2505.04568/) (2025)

## Built on by (most cited first, 4 of 4)

- [Online Safety Monitoring for LLMs](https://mlsanity.com/paper/2607.02510/) (2026)
- [TRACE: Trajectory Aware Reasoning for Multi-Turn Adversarial Conversation Evaluation](https://mlsanity.com/paper/2608.15594/) (2026)
- [Sharp Limits for Honest Uncertainty in Hard-Budget Repeated Evaluation](https://mlsanity.com/paper/2609.29140/) (2026)
- [Dynamic Budget Allocation for LLM Evaluation under Hard Resource Constraints](https://mlsanity.com/paper/2610.07362/) (2026)

## Abstract (arXiv)

Evaluating and predicting the performance of large language models (LLMs) in multi-turn conversational settings is critical yet computationally expensive; key events---e.g., jailbreaks or successful task completion by an agent---often emerge only after repeated interactions. These events might be rare and, under any feasible computational budget, remain unobserved. Recent conformal survival frameworks construct reliable lower predictive bounds (LPBs) on the number of iterations to trigger the event of interest, but rely on static budget allocation that prohibits adaptivity in multi-turn setups. To address this, we introduce Dynamic Allocation via PRojected Optimization (DAPRO), a theoretically valid dynamic budget allocation framework for bounding the time-to-event in multi-turn LLM interactions. We prove that DAPRO satisfies the expected budget constraint and provides distribution-free, finite-sample coverage guarantees without requiring the conditional independence between censoring and event times assumed by prior conformal survival approaches. A key theoretical contribution is a novel coverage bound that scales with the square root of the mean censoring weight over only a subset of the samples rather than the worst-case weight, yielding tighter guarantees than prior work. Furthermore, DAPRO can be employed to obtain unbiased estimates of population-level evaluation metrics, such as the jailbreak rate, under limited computing resources. Comprehensive experiments across agentic task success, LLM jailbreaks, toxic content generation, and RAG hallucinations demonstrate that DAPRO achieves coverage closer to the nominal level with lower variance than static baselines, while not exceeding the budget constraint in expectation.