# MRJ-Agent: An Effective Jailbreak Agent for Multi-Round Dialogue

arXiv: https://arxiv.org/abs/2411.03814 · HTML: https://mlsanity.com/paper/2411.03814/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use.
- Builds on 25 of its 35 references that are in the map.
- First posted on arXiv 6 Nov 2024.

## Placed in

- LLM agents and tool use / [Agent threats, guardrails and governance](https://mlsanity.com/agents/agent-threats-guardrails-and-governance/)
- LLM safety and security / [Jailbreaking](https://mlsanity.com/safety/jailbreaking/)

## Builds on (most cited first, 25 of 25)

- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Universal and Transferable Adversarial Attacks on Aligned Language Models](https://mlsanity.com/paper/2307.15043/) (2023)
- [Jailbreaking Black Box Large Language Models in Twenty Queries](https://mlsanity.com/paper/2310.08419/) (2023)
- [Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations](https://mlsanity.com/paper/2312.06674/) (2023)
- [Towards Understanding Sycophancy in Language Models](https://mlsanity.com/paper/2310.13548/) (2023)
- [Red Teaming Language Models with Language Models](https://mlsanity.com/paper/2202.03286/) (2022)
- [Tree of Attacks: Jailbreaking Black-Box LLMs Automatically](https://mlsanity.com/paper/2312.02119/) (2023)
- [Baseline Defenses for Adversarial Attacks Against Aligned Language Models](https://mlsanity.com/paper/2309.00614/) (2023)
- [How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMs](https://mlsanity.com/paper/2401.06373/) (2024)
- [GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Prompts](https://mlsanity.com/paper/2309.10253/) (2023)
- [Detecting Language Model Attacks with Perplexity](https://mlsanity.com/paper/2308.14132/) (2023)
- [GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Cipher](https://mlsanity.com/paper/2308.06463/) (2023)
- [Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attack](https://mlsanity.com/paper/2404.01833/) (2024)
- [Recursively Summarizing Books with Human Feedback](https://mlsanity.com/paper/2109.10862/) (2021)
- [Exploiting Programmatic Behavior of LLMs: Dual-Use Through Standard Security Attacks](https://mlsanity.com/paper/2302.05733/) (2023)
- [ArtPrompt: ASCII Art-based Jailbreak Attacks against Aligned LLMs](https://mlsanity.com/paper/2402.11753/) (2024)
- [COLD-Attack: Jailbreaking LLMs with Stealthiness and Controllability](https://mlsanity.com/paper/2402.08679/) (2024)
- [AdvPrompter: Fast Adaptive Adversarial Prompting for LLMs](https://mlsanity.com/paper/2404.16873/) (2024)
- [Chain of Attack: a Semantic-Driven Contextual Multi-Turn attacker for LLM](https://mlsanity.com/paper/2405.05610/) (2024)
- [Speak Out of Turn: Safety Vulnerability of Large Language Models in Multi-turn Dialogue](https://mlsanity.com/paper/2402.17262/) (2024)
- [Accelerating Greedy Coordinate Gradient and General Prompt Optimization via Probe Sampling](https://mlsanity.com/paper/2403.01251/) (2024)
- [Red Teaming GPT-4V: Are GPT-4V Safe Against Uni/Multi-Modal Jailbreak Attacks?](https://mlsanity.com/paper/2404.03411/) (2024)
- [Evolving Diverse Red-team Language Models in Multi-round Multi-agent Games](https://mlsanity.com/paper/2310.00322/) (2023)

## Built on by (most cited first, 30 of 34)

- [X-Teaming: Multi-Turn Jailbreaks and Defenses with Adaptive Multi-Agents](https://mlsanity.com/paper/2504.13203/) (2025)
- [A Survey on Trustworthy LLM Agents: Threats and Countermeasures](https://mlsanity.com/paper/2503.09648/) (2025)
- [Jailbreaking Multimodal Large Language Models via Shuffle Inconsistency](https://mlsanity.com/paper/2501.04931/) (2025)
- [MTSA: Multi-turn Safety Alignment for LLMs through Multi-round Red-teaming](https://mlsanity.com/paper/2505.17147/) (2025)
- [X-Boundary: Establishing Exact Safety Boundary to Shield LLMs from Multi-Turn Jailbreaks without Compromising Usability](https://mlsanity.com/paper/2502.09990/) (2025)
- [Steering Dialogue Dynamics for Robustness against Multi-turn Jailbreaking Attacks](https://mlsanity.com/paper/2503.00187/) (2025)
- [Inverse Reinforcement Learning with Dynamic Reward Scaling for LLM Alignment](https://mlsanity.com/paper/2503.18991/) (2025)
- [PBI-Attack: Prior-Guided Bimodal Interactive Black-Box Jailbreak Attack for Toxicity Maximization](https://mlsanity.com/paper/2412.05892/) (2024)
- [TROJail: Trajectory-Level Optimization for Multi-Turn Large Language Model Jailbreaks with Process Rewards](https://mlsanity.com/paper/2512.07761/) (2025)
- [From LLMs to MLLMs to Agents: A Survey of Emerging Paradigms in Jailbreak Attacks and Defenses within LLM Ecosystem](https://mlsanity.com/paper/2506.15170/) (2025)
- [SEMA: Simple yet Effective Learning for Multi-Turn Jailbreak Attacks](https://mlsanity.com/paper/2602.06854/) (2026)
- [Enhancing Model Defense Against Jailbreaks with Proactive Safety Reasoning](https://mlsanity.com/paper/2501.19180/) (2025)
- [Strata-Sword: A Hierarchical Safety Evaluation towards LLMs based on Reasoning Complexity of Jailbreak Instructions](https://mlsanity.com/paper/2509.01444/) (2025)
- [Cannot See the Forest for the Trees: Invoking Heuristics and Biases to Elicit Irrational Choices of LLMs](https://mlsanity.com/paper/2505.02862/) (2025)
- [Knowledge-Driven Multi-Turn Jailbreaking on Large Language Models](https://mlsanity.com/paper/2601.05445/) (2026)
- [VoxSafeBench: Not Just What Is Said, but Who, How, and Where](https://mlsanity.com/paper/2604.14548/) (2026)
- [MultiBreak: A Scalable and Diverse Multi-turn Jailbreak Benchmark for Evaluating LLM Safety](https://mlsanity.com/paper/2605.01687/) (2026)
- [NEXUS: Network Exploration for eXploiting Unsafe Sequences in Multi-Turn LLM Jailbreaks](https://mlsanity.com/paper/2510.03417/) (2025)
- [Learning to Conceal Risk: Controllable Multi-turn Red Teaming for LLMs in the Financial Domain](https://mlsanity.com/paper/2509.10546/) (2025)
- [Chain-of-Lure: A Universal Jailbreak Attack Framework using Unconstrained Synthetic Narratives](https://mlsanity.com/paper/2505.17519/) (2025)
- [HarmNet: A Framework for Adaptive Multi-Turn Jailbreak Attacks on Large Language Models](https://mlsanity.com/paper/2510.18728/) (2025)
- [VRSA: Jailbreaking Multimodal Large Language Models through Visual Reasoning Sequential Attack](https://mlsanity.com/paper/2512.05853/) (2025)
- [The Echo Chamber Multi-Turn LLM Jailbreak](https://mlsanity.com/paper/2601.05742/) (2026)
- [Attention-Aware GNN-based Input Defense against Multi-Turn LLM Jailbreak](https://mlsanity.com/paper/2507.07146/) (2025)
- [Multi-turn Jailbreaking Attack in Multi-Modal Large Language Models](https://mlsanity.com/paper/2601.05339/) (2026)
- [SoK: Robustness in Large Language Models against Jailbreak Attacks](https://mlsanity.com/paper/2605.05058/) (2026)
- [GRAF: Multi-turn Jailbreaking via Global Refinement and Active Fabrication](https://mlsanity.com/paper/2506.17881/) (2025)
- [Active Honeypot Guardrail System: Probing and Confirming Multi-Turn LLM Jailbreaks](https://mlsanity.com/paper/2510.15017/) (2025)
- [Jailbreaking Large Vision Language Models in Intelligent Transportation Systems](https://mlsanity.com/paper/2511.13892/) (2025)
- [Into the Gray Zone: Domain Contexts Can Blur LLM Safety Boundaries](https://mlsanity.com/paper/2604.15717/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) demonstrate outstanding performance in their reservoir of knowledge and understanding capabilities, but they have also been shown to be prone to illegal or unethical reactions when subjected to jailbreak attacks. To ensure their responsible deployment in critical applications, it is crucial to understand the safety capabilities and vulnerabilities of LLMs. Previous works mainly focus on jailbreak in single-round dialogue, overlooking the potential jailbreak risks in multi-round dialogues, which are a vital way humans interact with and extract information from LLMs. Some studies have increasingly concentrated on the risks associated with jailbreak in multi-round dialogues. These efforts typically involve the use of manually crafted templates or prompt engineering techniques. However, due to the inherent complexity of multi-round dialogues, their jailbreak performance is limited. To solve this problem, we propose a novel multi-round dialogue jailbreaking agent, emphasizing the importance of stealthiness in identifying and mitigating potential threats to human values posed by LLMs. We propose a risk decomposition strategy that distributes risks across multiple rounds of queries and utilizes psychological strategies to enhance attack strength. Extensive experiments show that our proposed method surpasses other attack methods and achieves state-of-the-art attack success rate. We will make the corresponding code and dataset available for future research. The code will be released soon.