SafeDream: Safety World Model for Proactive Early Jailbreak Detection
On the map: World models → World model security and GUI agents
also in 2 more
Abstract (arXiv)
Multi-turn jailbreak attacks progressively erode LLM safety alignment across seemingly innocuous conversation turns, achieving success rates exceeding 90% against state-of-the-art models. Existing alignment-based and guardrail methods suffer from three key limitations: they require costly weight modification, evaluate each turn independently without modeling cumulative safety erosion, and detect attacks only after harmful content has been generated. To address these limitations, we first formulate the proactive early jailbreak detection problem with a new metric, detection lead, that measures how early an attack can be detected before the LLM complies. We then propose SAFEDREAM, a lightweight world-model-based framework that operates as an external module without modifying the LLM's weights. SAFEDREAM introduces three components: (1) a safety state world model that encodes LLM hidden states into a compact safety representation and predicts how it evolves across turns, (2) CUSUM detection that accumulates weak per-turn risk signals into reliable evidence, and (3) contrastive imagination that simultaneously rolls out attack and benign futures in latent space to issue early alarms before jailbreaks occur. On three multi-turn jailbreak benchmarks (XGuard-Train, SafeDialBench, SafeMTData) against 8 baselines, SAFEDREAM achieves the best detection timeliness across all benchmarks (1.06-1.20 turns before compliance) while maintaining competitive false positive rates and outperforming baselines in detection quality.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in World model security and GUI agents, one of 11 directions in World models. Builds on 24 of its 31 references that are in the map. First posted on arXiv 18 Apr 2026.
Builds on
24 of 31 references.
References · 24
- 2024Qwen2.5 Technical Reportcited by 4,713
- 2024GPT-4o System Cardcited by 4,467
- 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
- 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,124
- 2024Refusal in Language Models Is Mediated by a Single Directioncited by 960
- 2021Perceiver IO: A General Architecture for Structured Inputs & Outputscited by 616
- 2024WildGuard: Open One-Stop Moderation Tools for Safety Risks, Jailbreaks, and Refusals of LLMscited by 460
- 2024Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attackcited by 364
- 2024Improving Alignment and Robustness with Circuit Breakerscited by 302
- 2024SALAD-Bench: A Hierarchical and Comprehensive Safety Benchmark for Large Language Modelscited by 225
- 2025X-Teaming: Multi-Turn Jailbreaks and Defenses with Adaptive Multi-Agentscited by 112
- 2025The Geometry of Refusal in Large Language Models: Concept Cones and Representational Independencecited by 104
12 more
- 2025GuardReasoner: Towards Reasoning-based LLM Safeguardscited by 89
- 2024LLMs know their vulnerabilities: Uncover Safety Gaps through Natural Distribution Shiftscited by 80
- 2024SelfDefend: LLMs Can Defend Themselves against Jailbreaking in a Practical Mannercited by 57
- 2025Foot-In-The-Door: A Multi-turn Jailbreak for LLMscited by 50
- 2024R2-Guard: Robust Reasoning Enabled LLM Guardrail via Knowledge-Enhanced Logical Reasoningcited by 41
- 2025Next-Latent Prediction Transformers Learn Compact World Modelscited by 29
- 2024JailbreakEval: An Integrated Toolkit for Evaluating Jailbreak Attempts Against Large Language Modelscited by 28
- 2025SafeDialBench: A Fine-Grained Safety Evaluation Benchmark for Large Language Models in Multi-Turn Dialogues with Diverse Jailbreak Attackscited by 23
- 2025Reasoning as an Adaptive Defense for Safetycited by 22
- 2025BingoGuard: LLM Content Moderation Tools with Risk Levelscited by 20
- 2025Advancing LLM Safe Alignment with Safety Representation Rankingcited by 15
- 2026State-Dependent Safety Failures in Multi-Turn Language Model Interactioncited by 4
Built on by
Citing papers per year: 2026: 2.
2 papers.
- 2026Agentic World Modeling: Foundations, Capabilities, Laws, and Beyondcited by 10
- 2026Quo Vadis, World Modeling?cited by 0
- 2026Quo Vadis, World Modeling?3 Aug 2026
- 2026Agentic World Modeling: Foundations, Capabilities, Laws, and Beyond24 Apr 2026
Similar, not cited
Close in wording, no citation either way
- 2026MultiBreak: A Scalable and Diverse Multi-turn Jailbreak Benchmark for Evaluating LLM Safety
- 2026Defending Large Language Models Against Jailbreak Attacks via In-Decoding Safety-Awareness Probing
- 2024SafeAligner: Safety Alignment against Jailbreak Attacks via Response Disparity Guidance
- 2025Forewarned is Forearmed: Pre-Synthesizing Jailbreak-like Instructions to Enhance LLM Safety Guardrail to Potential Attacks
- 2025Jailbreaking in the Haystack
- 2026Exploring and Developing a Pre-Model Safeguard with Draft Models
- 2025PandaGuard: Systematic Evaluation of LLM Safety against Jailbreaking Attacks
- 2024How Alignment and Jailbreak Work: Explain LLM Safety through Intermediate Hidden States