DreamGuard: Efficient Runtime Guardrail for LLM Agents via Risk-Aware World Model
On the map: World models → World model security and GUI agents
also in 2 more
Abstract (arXiv)
As large language model (LLM) agents increasingly invoke external tools and interact with real-world systems, unsafe actions may cause irreversible consequences on external states, user data, and downstream services. Recent runtime guardrails mitigate such risks by checking proposed actions before execution, but many remain reactive: they primarily assess the apparent safety of the current action, lacking an explicit model of how risk evolves across the trajectory. This limitation creates a critical blind spot for long-horizon risks, where individually benign-looking actions can gradually drift the agent toward hazardous states. In response, we propose DreamGuard, a proactive guardrail for LLM agents built around a risk-aware world model. The world model maintains a compact recurrent latent state over the trajectory and predicts future latent states from which DreamGuard derives immediate-hazard and prefix-risk evidence. It then fuses these multi-horizon signals into intervention decisions before execution. Experiments across four benchmarks and an online guardrail evaluation show that DreamGuard outperforms generic, reactive, and proactive guardrail baselines, achieves the best safety-utility trade-off among evaluated guardrails, and maintains an average end-to-end latency of 25 ms per call.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in World model security and GUI agents, one of 11 directions in World models. Builds on 20 of its 29 references that are in the map. First posted on arXiv 6 Aug 2026.
Builds on
20 of 29 references.
Closest ancestors · 1
Same direction, newest first
Other references · 19
- 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
- 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
- 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
- 2018Learning Latent Dynamics for Planning from Pixelscited by 1,221
- 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,124
- 2023TD-MPC2: Scalable, Robust World Models for Continuous Controlcited by 482
- 2024Agent Security Bench (ASB): Formalizing and Benchmarking Attacks and Defenses in LLM-based Agentscited by 341
- 2024Is Your LLM Secretly a World Model of the Internet? Model-Based Planning for Web Agentscited by 144
- 2025ShieldAgent: Shielding Agents via Verifiable Safety Policy Reasoningcited by 144
- 2024GuardAgent: Safeguard LLM Agents by a Guard Agent via Knowledge-Enabled Reasoningcited by 113
- 2025AGrail: A Lifelong Agent Guardrail with Effective and Adaptive Safety Detectioncited by 82
- 2025AgentAuditor: Human-Level Safety and Security Evaluation for LLM Agentscited by 82
7 more
- 2026ToolSafe: Enhancing Tool Invocation Safety of LLM-based agents via Proactive Step-level Guardrail and Feedbackcited by 64
- 2025Safety at Scale: A Comprehensive Survey of Large Model and Agent Safetycited by 63
- 2026SafeAgent: A Runtime Protection Architecture for Agentic Systemscited by 7
- 2026Safeguarding LLM Agents against Long-Horizon Threats via Shadow Memorycited by 6
- 2026TRACES: Proactive Safety Auditing for Multi-Turn LLM Agents via Trajectory-State Modelingcited by 5
- 2026SafetyDrift: Predicting When AI Agents Cross the Line Before They Actually Docited by 4
- 2026SafeMCP: Proactive Power Regulation for LLM Agent Defense via Environment-Grounded Look-Ahead Reasoningcited by 3
Built on by
Citing papers per year: 2026: 3.
3 papers.
- 2026Adaptive Triggering for Bias Correction in LLM Reasoningcited by 0
- 2026DriftNet: A Dual-Head Trajectory Transformer for Detecting and Localizing Prompt Injection in LLM Agentscited by 0
- 2026Safe Actions Alone Do Not Ensure Safe Agents: Identifying Unfulfilled Obligations with Guard Modelscited by 0
- 2026Safe Actions Alone Do Not Ensure Safe Agents: Identifying Unfulfilled Obligations with Guard Models8 Oct 2026
- 2026DriftNet: A Dual-Head Trajectory Transformer for Detecting and Localizing Prompt Injection in LLM Agents9 Sep 2026
- 2026Adaptive Triggering for Bias Correction in LLM Reasoning26 Aug 2026
Similar, not cited
Close in wording, no citation either way
- 2025ProbGuard: Proactive Runtime Monitoring for LLM Agent Safety via Probabilistic Prediction
- 2026From Risk Classification to Action Plan Remediation: A Guardrail Feedback Driven Framework for LLM Agents
- 2025WebGuard: Building a Generalizable Guardrail for Web Agents
- 2025Building a Foundational Guardrail for General Agentic Systems via Synthetic Data
- 2026EMBGuard: Constructing Hazard-Aware Guardrails for Safe Planning in Embodied Agents
- 2025LlamaFirewall: An open source guardrail system for building secure AI agents
- 2026BraveGuard: From Open-World Threats to Safer Computer-Use Agents
- 2026TraceSafe: A Systematic Assessment of LLM Guardrails on Multi-Step Tool-Calling Trajectories