# Jailbreak-R1: Exploring the Jailbreak Capabilities of LLMs via Reinforcement Learning

arXiv: https://arxiv.org/abs/2506.00782 · HTML: https://mlsanity.com/paper/2506.00782/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Placed in Prompt injection and agent security, one of 12 directions in LLM safety and security.
- First posted on arXiv 1 Jun 2025.

## Placed in

- LLM safety and security / [Prompt injection and agent security](https://mlsanity.com/safety/prompt-injection-and-agent-security/)

## Built on by (most cited first, 27 of 27)

- [RL Is a Hammer and LLMs Are Nails: A Simple Reinforcement Learning Recipe for Strong Prompt Injection](https://mlsanity.com/paper/2510.04885/) (2025)
- [SEMA: Simple yet Effective Learning for Multi-Turn Jailbreak Attacks](https://mlsanity.com/paper/2602.06854/) (2026)
- [Tree-based Dialogue Reinforced Policy Optimization for Red-Teaming Attacks](https://mlsanity.com/paper/2510.02286/) (2025)
- [The Trojan Knowledge: Bypassing Commercial LLM Guardrails via Harmless Prompt Weaving and Adaptive Tree Search](https://mlsanity.com/paper/2512.01353/) (2025)
- [Be Your Own Red Teamer: Safety Alignment via Self-Play and Reflective Experience Replay](https://mlsanity.com/paper/2601.10589/) (2026)
- [Statistical Estimation of Adversarial Risk in Large Language Models under Best-of-N Sampling](https://mlsanity.com/paper/2601.22636/) (2026)
- [SIRAJ: Diverse and Efficient Red-Teaming for LLM Agents via Distilled Structured Reasoning](https://mlsanity.com/paper/2510.26037/) (2025)
- [AgenticRed: Evolving Agentic Systems for Red-Teaming](https://mlsanity.com/paper/2601.13518/) (2026)
- [Learning to Attack and Defend: Adaptive Red Teaming of Language Models via GRPO](https://mlsanity.com/paper/2606.09701/) (2026)
- [Not All Turns Matter: Credit Assignment for Multi-Turn Jailbreaking](https://mlsanity.com/paper/2605.08778/) (2026)
- [Internalizing Safety Understanding in Large Reasoning Models via Verification](https://mlsanity.com/paper/2605.08930/) (2026)
- [Activation Surgery: Jailbreaking White-box LLMs without Touching the Prompt](https://mlsanity.com/paper/2603.14278/) (2026)
- [GFlowRL: Scaling Distribution-Matching RL to Large Language Models](https://mlsanity.com/paper/2607.13394/) (2026)
- [Learning to Persuade Exposes How Easily LLMs Abandon Correct Beliefs](https://mlsanity.com/paper/2608.11624/) (2026)
- [Why Does Weak-OOD Help? A Further Step Towards Understanding Jailbreaking VLMs](https://mlsanity.com/paper/2511.08367/) (2025)
- [David vs. Goliath: Verifiable Agent-to-Agent Jailbreaking via Reinforcement Learning](https://mlsanity.com/paper/2602.02395/) (2026)
- [Stable-GFlowNet: Toward Diverse and Robust LLM Red-Teaming via Contrastive Trajectory Balance](https://mlsanity.com/paper/2605.00553/) (2026)
- [Self-ReSET: Learning to Self-Recover from Unsafe Reasoning Trajectories](https://mlsanity.com/paper/2605.08936/) (2026)
- [Skill Weaving: Efficient LLM Improvement via Modular Skillpacks](https://mlsanity.com/paper/2605.22205/) (2026)
- [An Early Warning of Emerging Biosecurity Risks in Frontier LLMs](https://mlsanity.com/paper/2607.18056/) (2026)
- [Safin-1: Safety from Within through Memory-Native State Evolution](https://mlsanity.com/paper/2609.00092/) (2026)
- [A Systematic Investigation of RL-Jailbreaking in LLMs](https://mlsanity.com/paper/2605.07032/) (2026)
- [Model-Agnostic Lifelong LLM Safety via Externalized Attack-Defense Co-Evolution](https://mlsanity.com/paper/2605.13411/) (2026)
- [All you need to break LLMs are Black-Box, Adapting, Efficient, Transferable, Harmful, Applicable ... Attacks](https://mlsanity.com/paper/2606.03647/) (2026)
- [Refused in Chat, Written in Code: Workflow-Level Jailbreak Construction in IDE Coding Agents](https://mlsanity.com/paper/2607.03968/) (2026)
- [MJ: Multi-turn LLM Jailbreaking via Decomposed Credit Assignment](https://mlsanity.com/paper/2607.11070/) (2026)
- [Red-TTT: Test-Time Training for Automated Jailbreaking Large Language Models](https://mlsanity.com/paper/2610.05282/) (2026)

## Abstract (arXiv)

As large language models (LLMs) grow in power and influence, ensuring their safety and preventing harmful output becomes critical. Automated red teaming serves as a tool to detect security vulnerabilities in LLMs without manual labor. However, most existing methods struggle to balance the effectiveness and diversity of red-team generated attack prompts. To address this challenge, we propose ourapproach, a novel automated red teaming training framework that utilizes reinforcement learning to explore and generate more effective attack prompts while balancing their diversity. Specifically, it consists of three training stages: (1) Cold Start: The red team model is supervised and fine-tuned on a jailbreak dataset obtained through imitation learning. (2) Warm-up Exploration: The model is trained in jailbreak instruction following and exploration, using diversity and consistency as reward signals. (3) Enhanced Jailbreak: Progressive jailbreak rewards are introduced to gradually enhance the jailbreak performance of the red-team model. Extensive experiments on a variety of LLMs show that ourapproach effectively balances the diversity and effectiveness of jailbreak prompts compared to existing methods. Our work significantly improves the efficiency of red team exploration and provides a new perspective on automated red teaming.