ContextualJailbreak: Evolutionary Red-Teaming via Simulated Conversational Priming
On the map: LLM alignment / RLHF / preference optimisation → Safety alignment and jailbreak attacks
also in 1 more
Abstract (arXiv)
Large language models (LLMs) remain vulnerable to jailbreak attacks that bypass safety alignment and elicit harmful responses. A growing body of work shows that contextual priming, where earlier turns covertly bias later replies, constitutes a powerful attack surface, with hand-crafted multi-turn scaffolds consistently outperforming single-turn manipulations on capable models. However, automated optimization-based red-teaming has remained largely limited to the single-turn setting, iterating over static prompts and lacking the ability to reason about which forms of conversational priming induce compliance. While recent multi-turn, search-based approaches have begun to bridge this gap, the mutator design space underlying effective primed dialogues remains largely unexplored. We present ContextualJailbreak, a black-box red-teaming strategy that performs evolutionary search over a simulated multi-turn primed dialogue. The strategy leverages a graded 0-5 harm score from a two-level judge as an in-loop signal, enabling partially harmful responses to guide the search process rather than being discarded. Search is driven by five semantically defined mutation operators: roleplay, scenario, expand, troubleshooting, and mechanistic, of which the last two are novel contributions of this work. Across 50 representative HarmBench behaviors, ContextualJailbreak achieves an ASR of 100% on gpt-oss:20B, 100% on qwen3-8B, 100% on llama3.1:70B, and 90% on gpt-oss:120B, outperforming four single- and multi-turn baselines by 31-96 percentage points on average. The 40 maximally harmful attacks discovered against gpt-oss:120B transfer without adaptation to closed frontier models, achieving 90.0% on gpt-4o-mini, 70.0% on gpt-5, and 70.0% on gemini-3-flash, but only 17.5% on claude-opus-4-7 and 15.0% on claude-sonnet-4-6, revealing a pronounced provider-level asymmetry in alignment robustness.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Safety alignment and jailbreak attacks, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 17 of its 38 references that are in the map. First posted on arXiv 4 May 2026.
Builds on
17 of 38 references.
Closest ancestors · 1
Same direction, newest first
Other references · 16
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
- 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
- 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,326
- 2022Ignore Previous Prompt: Attack Techniques For Language Modelscited by 760
- 2023Tree of Attacks: Jailbreaking Black-Box LLMs Automaticallycited by 619
- 2023GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Promptscited by 520
- 2024Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attackcited by 364
- 2024Rainbow Teaming: Open-Ended Generation of Diverse Adversarial Promptscited by 189
- 2024AutoDAN-Turbo: A Lifelong Agent for Strategy Self-Exploration to Jailbreak LLMscited by 185
- 2025Foot-In-The-Door: A Multi-turn Jailbreak for LLMscited by 50
- 2019apricot: Submodular selection for data summarization in Pythoncited by 21
4 more
- 2024PAPILLON: Efficient and Stealthy Fuzz Testing-Powered Jailbreaks for LLMscited by 20
- 2025BadJudge: Backdoor Vulnerabilities of LLM-as-a-Judgecited by 13
- 2025Large Reasoning Models Are Autonomous Jailbreak Agentscited by 12
- 2025Response Attack: Exploiting Contextual Priming to Jailbreak Large Language Modelscited by 10
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2025MUSE: MCTS-Driven Red Teaming Framework for Enhanced Multi-Turn Dialogue Safety in Large Language Models
- 2026AutoRISE: Agent-Driven Strategy Evolution for Red-Teaming Large Language Models
- 2025ToxSearch: Evolving Prompts for Toxicity Search in Large Language Models
- 2025PLAGUE: Plug-and-play framework for Lifelong Adaptive Generation of Multi-turn Exploits
- 2025RedTopic: Toward Topic-Diverse Red Teaming of Large Language Models
- 2025Text-Diffusion Red-Teaming of Large Language Models: Unveiling Harmful Behaviors with Proximity Constraints
- 2025RedDebate: Safer Responses Through Multi-Agent Red Teaming Debates
- 2026CART: Closed-Loop Adaptive Red Teaming for Large Language Models