mlsanityData to 7 Oct 2026

ContextualJailbreak: Evolutionary Red-Teaming via Simulated Conversational Priming

arXiv 2605.02647 ↗First posted 4 May 2026cs.CL cs.CRcited by 1

On the map: LLM alignment / RLHF / preference optimisation → Safety alignment and jailbreak attacks

also in 1 more
Abstract (arXiv)

Large language models (LLMs) remain vulnerable to jailbreak attacks that bypass safety alignment and elicit harmful responses. A growing body of work shows that contextual priming, where earlier turns covertly bias later replies, constitutes a powerful attack surface, with hand-crafted multi-turn scaffolds consistently outperforming single-turn manipulations on capable models. However, automated optimization-based red-teaming has remained largely limited to the single-turn setting, iterating over static prompts and lacking the ability to reason about which forms of conversational priming induce compliance. While recent multi-turn, search-based approaches have begun to bridge this gap, the mutator design space underlying effective primed dialogues remains largely unexplored. We present ContextualJailbreak, a black-box red-teaming strategy that performs evolutionary search over a simulated multi-turn primed dialogue. The strategy leverages a graded 0-5 harm score from a two-level judge as an in-loop signal, enabling partially harmful responses to guide the search process rather than being discarded. Search is driven by five semantically defined mutation operators: roleplay, scenario, expand, troubleshooting, and mechanistic, of which the last two are novel contributions of this work. Across 50 representative HarmBench behaviors, ContextualJailbreak achieves an ASR of 100% on gpt-oss:20B, 100% on qwen3-8B, 100% on llama3.1:70B, and 90% on gpt-oss:120B, outperforming four single- and multi-turn baselines by 31-96 percentage points on average. The 40 maximally harmful attacks discovered against gpt-oss:120B transfer without adaptation to closed frontier models, achieving 90.0% on gpt-4o-mini, 70.0% on gpt-5, and 70.0% on gemini-3-flash, but only 17.5% on claude-opus-4-7 and 15.0% on claude-sonnet-4-6, revealing a pronounced provider-level asymmetry in alignment robustness.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Safety alignment and jailbreak attacks, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 17 of its 38 references that are in the map. First posted on arXiv 4 May 2026.

Builds on

17 of 38 references.

Closest ancestors · 1

Same direction, newest first

  1. 2026Into the Gray Zone: Domain Contexts Can Blur LLM Safety Boundaries17 Apr

Other references · 16

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  3. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
  4. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,326
  5. 2022Ignore Previous Prompt: Attack Techniques For Language Modelscited by 760
  6. 2023Tree of Attacks: Jailbreaking Black-Box LLMs Automaticallycited by 619
  7. 2023GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Promptscited by 520
  8. 2024Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attackcited by 364
  9. 2024Rainbow Teaming: Open-Ended Generation of Diverse Adversarial Promptscited by 189
  10. 2024AutoDAN-Turbo: A Lifelong Agent for Strategy Self-Exploration to Jailbreak LLMscited by 185
  11. 2025Foot-In-The-Door: A Multi-turn Jailbreak for LLMscited by 50
  12. 2019apricot: Submodular selection for data summarization in Pythoncited by 21
4 more
  1. 2024PAPILLON: Efficient and Stealthy Fuzz Testing-Powered Jailbreaks for LLMscited by 20
  2. 2025BadJudge: Backdoor Vulnerabilities of LLM-as-a-Judgecited by 13
  3. 2025Large Reasoning Models Are Autonomous Jailbreak Agentscited by 12
  4. 2025Response Attack: Exploiting Contextual Priming to Jailbreak Large Language Modelscited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2025MUSE: MCTS-Driven Red Teaming Framework for Enhanced Multi-Turn Dialogue Safety in Large Language Models
  2. 2026AutoRISE: Agent-Driven Strategy Evolution for Red-Teaming Large Language Models
  3. 2025ToxSearch: Evolving Prompts for Toxicity Search in Large Language Models
  4. 2025PLAGUE: Plug-and-play framework for Lifelong Adaptive Generation of Multi-turn Exploits
  5. 2025RedTopic: Toward Topic-Diverse Red Teaming of Large Language Models
  6. 2025Text-Diffusion Red-Teaming of Large Language Models: Unveiling Harmful Behaviors with Proximity Constraints
  7. 2025RedDebate: Safer Responses Through Multi-Agent Red Teaming Debates
  8. 2026CART: Closed-Loop Adaptive Red Teaming for Large Language Models