mlsanityData to 14 Oct 2026

MRJ-Agent: An Effective Jailbreak Agent for Multi-Round Dialogue

arXiv 2411.03814 ↗First posted 6 Nov 2024cs.AI cs.CL cs.CRcited by 33

On the map: LLM agents and tool use → Agent threats, guardrails and governance

also in 1 more
Abstract (arXiv)

Large Language Models (LLMs) demonstrate outstanding performance in their reservoir of knowledge and understanding capabilities, but they have also been shown to be prone to illegal or unethical reactions when subjected to jailbreak attacks. To ensure their responsible deployment in critical applications, it is crucial to understand the safety capabilities and vulnerabilities of LLMs. Previous works mainly focus on jailbreak in single-round dialogue, overlooking the potential jailbreak risks in multi-round dialogues, which are a vital way humans interact with and extract information from LLMs. Some studies have increasingly concentrated on the risks associated with jailbreak in multi-round dialogues. These efforts typically involve the use of manually crafted templates or prompt engineering techniques. However, due to the inherent complexity of multi-round dialogues, their jailbreak performance is limited. To solve this problem, we propose a novel multi-round dialogue jailbreaking agent, emphasizing the importance of stealthiness in identifying and mitigating potential threats to human values posed by LLMs. We propose a risk decomposition strategy that distributes risks across multiple rounds of queries and utilizes psychological strategies to enhance attack strength. Extensive experiments show that our proposed method surpasses other attack methods and achieves state-of-the-art attack success rate. We will make the corresponding code and dataset available for future research. The code will be released soon.

Key facts. Cited by 33 papers in the map (data through 14 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 25 of its 35 references that are in the map. First posted on arXiv 6 Nov 2024.

Builds on

25 of 35 references.

References · 25

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  4. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  5. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,420
  6. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,207
  7. 2023Towards Understanding Sycophancy in Language Modelscited by 1,040
  8. 2022Red Teaming Language Models with Language Modelscited by 1,026
  9. 2023Tree of Attacks: Jailbreaking Black-Box LLMs Automaticallycited by 672
  10. 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Modelscited by 651
  11. 2024How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMscited by 573
  12. 2023GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Promptscited by 560
13 more
  1. 2023Detecting Language Model Attacks with Perplexitycited by 431
  2. 2023GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Ciphercited by 423
  3. 2024Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attackcited by 390
  4. 2021Recursively Summarizing Books with Human Feedbackcited by 290
  5. 2023Exploiting Programmatic Behavior of LLMs: Dual-Use Through Standard Security Attackscited by 278
  6. 2024ArtPrompt: ASCII Art-based Jailbreak Attacks against Aligned LLMscited by 216
  7. 2024COLD-Attack: Jailbreaking LLMs with Stealthiness and Controllabilitycited by 175
  8. 2024AdvPrompter: Fast Adaptive Adversarial Prompting for LLMscited by 156
  9. 2024Chain of Attack: a Semantic-Driven Contextual Multi-Turn attacker for LLMcited by 77
  10. 2024Speak Out of Turn: Safety Vulnerability of Large Language Models in Multi-turn Dialoguecited by 66
  11. 2024Accelerating Greedy Coordinate Gradient and General Prompt Optimization via Probe Samplingcited by 37
  12. 2024Red Teaming GPT-4V: Are GPT-4V Safe Against Uni/Multi-Modal Jailbreak Attacks?cited by 34
  13. 2023Evolving Diverse Red-team Language Models in Multi-round Multi-agent Gamescited by 17

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 22; 2026: 10.

33 papers.

Sort:

By area:

  1. 2025X-Teaming: Multi-Turn Jailbreaks and Defenses with Adaptive Multi-Agentscited by 125
  2. 2025A Survey on Trustworthy LLM Agents: Threats and Countermeasurescited by 96
  3. 2025Jailbreaking Multimodal Large Language Models via Shuffle Inconsistencycited by 51
  4. 2025MTSA: Multi-turn Safety Alignment for LLMs through Multi-round Red-teamingcited by 32
  5. 2025X-Boundary: Establishing Exact Safety Boundary to Shield LLMs from Multi-Turn Jailbreaks without Compromising Usabilitycited by 20
  6. 2025Steering Dialogue Dynamics for Robustness against Multi-turn Jailbreaking Attackscited by 20
  7. 2025Inverse Reinforcement Learning with Dynamic Reward Scaling for LLM Alignmentcited by 20
  8. 2024PBI-Attack: Prior-Guided Bimodal Interactive Black-Box Jailbreak Attack for Toxicity Maximizationcited by 17
  9. 2025TROJail: Trajectory-Level Optimization for Multi-Turn Large Language Model Jailbreaks with Process Rewardscited by 14
  10. 2025From LLMs to MLLMs to Agents: A Survey of Emerging Paradigms in Jailbreak Attacks and Defenses within LLM Ecosystemcited by 12
  11. 2026SEMA: Simple yet Effective Learning for Multi-Turn Jailbreak Attackscited by 12
  12. 2025Enhancing Model Defense Against Jailbreaks with Proactive Safety Reasoningcited by 11
21 more
  1. 2025Strata-Sword: A Hierarchical Safety Evaluation towards LLMs based on Reasoning Complexity of Jailbreak Instructionscited by 11
  2. 2025Cannot See the Forest for the Trees: Invoking Heuristics and Biases to Elicit Irrational Choices of LLMscited by 9
  3. 2026VoxSafeBench: Not Just What Is Said, but Who, How, and Wherecited by 8
  4. 2026MultiBreak: A Scalable and Diverse Multi-turn Jailbreak Benchmark for Evaluating LLM Safetycited by 8
  5. 2025NEXUS: Network Exploration for eXploiting Unsafe Sequences in Multi-Turn LLM Jailbreakscited by 7
  6. 2026Knowledge-Driven Multi-Turn Jailbreaking on Large Language Modelscited by 7
  7. 2025Learning to Conceal Risk: Controllable Multi-turn Red Teaming for LLMs in the Financial Domaincited by 6
  8. 2025Chain-of-Lure: A Universal Jailbreak Attack Framework using Unconstrained Synthetic Narrativescited by 5
  9. 2025VRSA: Jailbreaking Multimodal Large Language Models through Visual Reasoning Sequential Attackcited by 5
  10. 2025HarmNet: A Framework for Adaptive Multi-Turn Jailbreak Attacks on Large Language Modelscited by 4
  11. 2026The Echo Chamber Multi-Turn LLM Jailbreakcited by 4
  12. 2026Multi-turn Jailbreaking Attack in Multi-Modal Large Language Modelscited by 3
  13. 2026SoK: Robustness in Large Language Models against Jailbreak Attackscited by 3
  14. 2025GRAF: Multi-turn Jailbreaking via Global Refinement and Active Fabricationcited by 2
  15. 2025Attention-Aware GNN-based Input Defense against Multi-Turn LLM Jailbreakcited by 2
  16. 2025Active Honeypot Guardrail System: Probing and Confirming Multi-Turn LLM Jailbreakscited by 2
  17. 2025Jailbreaking Large Vision Language Models in Intelligent Transportation Systemscited by 1
  18. 2026Into the Gray Zone: Domain Contexts Can Blur LLM Safety Boundariescited by 1
  19. 2026SoK: Intent-Oriented Systematization of Multi-Turn LLM Jailbreakscited by 1
  20. 2025Pluralistic Behavior Suite: Stress-Testing Multi-Turn Adherence to Custom Behavioral Policiescited by 0
  21. 2026A Multimodal Automatic Redteaming Evaluation based on Atomic Jailbreak Strategy Decoupling and Combinationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Evaluating Multi-Agent Defences Against Jailbreaking Attacks on Large Language Models
  2. 2025Amplified Vulnerabilities: Structured Jailbreak Attacks on LLM-based Multi-Agent Debate
  3. 2025Automating Deception: Scalable Multi-Turn LLM Jailbreaks
  4. 2024Understanding Jailbreak Success: A Study of Latent Space Dynamics in Large Language Models
  5. 2023MasterKey: Automated Jailbreak Across Multiple Large Language Model Chatbots
  6. 2025Speak Easy: Eliciting Harmful Jailbreaks from LLMs with Simple Interactions
  7. 2025Dialogue Injection Attack: Jailbreaking LLMs through Context Manipulation
  8. 2024Jailbreaking and Mitigation of Vulnerabilities in Large Language Models