GuardAD: Safeguarding Autonomous Driving MLLMs via Markovian Safety Logic
On the map: End-to-end autonomous driving → Large language models for driving
Abstract (arXiv)
Multimodal large language models (MLLMs) are increasingly integrated into autonomous driving (AD) systems; however, they remain vulnerable to diverse safety threats, particularly in accident-prone scenarios. Recent safeguard mechanisms have shown promise by incorporating logical constraints, yet most rely on static formulations that lack temporally grounded safety reasoning over evolving traffic interactions, resulting in limited robustness in dynamic driving environments. To address these limitations, we propose GuardAD, a model-agnostic safeguard that formulates AD safety as an evolving Markovian logical state. GuardAD introduces Neuro-Symbolic Logic Formalization, which represents safety predicates over heterogeneous traffic participants and continuously induces them via n-th order Markovian Logic Induction. This design enables the inference of emerging and latent hazards beyond single-step observations. Rather than simply vetoing unsafe actions, GuardAD performs Logic-Driven Action Revision, where inferred safety states actively guide action refinement without modifying the underlying MLLM. Extensive experiments on multiple benchmarks and AD-MLLMs demonstrate that GuardAD substantially reduces accident rates (-32.07%) while slightly improving task performance (+6.85%). Moreover, closed-loop simulation evaluations, together with physical-world vehicle studies, further validate the effectiveness and potential of GuardAD.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Large language models for driving, one of 9 directions in End-to-end autonomous driving. Builds on 31 of its 41 references that are in the map. First posted on arXiv 11 May 2026.
Builds on
31 of 41 references.
Closest ancestors · 2
Same direction, newest first
- 2025Text2Scenario: Text-Driven Scenario Generation for Autonomous Driving Test4 Mar
- 2023DriveMLM: Aligning Multi-Modal Large Language Models with Behavioral Planning States for Autonomous Driving14 Dec
Other references · 29
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
- 2025AgentSpec: Customizable Runtime Enforcement for Safe and Reliable LLM Agentscited by 223
- 2025ORION: A Holistic End-to-End Autonomous Driving Framework by Vision-Language Instructed Action Generationcited by 178
- 2025OmniDrive: A Holistic Vision-Language Dataset for Autonomous Driving with Counterfactual Reasoningcited by 143
- 2023Dolphins: Multimodal Language Model for Drivingcited by 124
- 2024GuardAgent: Safeguard LLM Agents by a Guard Agent via Knowledge-Enabled Reasoningcited by 124
- 2024SafeAgentBench: A Benchmark for Safe Task Planning of Embodied LLM Agentscited by 116
- 2024Chain of Attack: a Semantic-Driven Contextual Multi-Turn attacker for LLMcited by 77
- 2020Dirty Road Can Attack: Security of Deep Learning based Automated Lane Centering under Physical-World Attackcited by 64
- 2024Code-as-Monitor: Constraint-aware Visual Programming for Reactive and Proactive Robotic Failure Detectioncited by 63
- 2023CAT: Closed-loop Adversarial Training for Safe End-to-End Drivingcited by 54
- 2024Multi-Frame, Lightweight & Efficient Vision-Language Models for Question Answering in Autonomous Drivingcited by 52
17 more
- 2025AGENTSAFE: Benchmarking the Safety of Embodied Agents on Hazardous Instructionscited by 38
- 2024Visual Adversarial Attack on Vision-Language Models for Autonomous Drivingcited by 36
- 2025RoboFactory: Exploring Embodied Agent Collaboration with Compositional Constraintscited by 31
- 2024Towards Robust Physical-world Backdoor Attacks on Lane Detectioncited by 26
- 2025Black-Box Adversarial Attack on Vision Language Models for Autonomous Drivingcited by 24
- 2025SafeAuto: Knowledge-Enhanced Safe Autonomous Driving with Multimodal Foundation Modelscited by 23
- 2025VRU-Accident: A Vision-Language Benchmark for Video Question Answering and Dense Captioning for Accident Scene Understandingcited by 20
- 2025S4-Driver: Scalable Self-Supervised Driving Multimodal Large Language Modelwith Spatio-Temporal Visual Representationcited by 19
- 2024LanEvil: Benchmarking the Robustness of Lane Detection to Environmental Illusionscited by 18
- 2025Natural Reflection Backdoor Attack on Vision Language Model for Autonomous Drivingcited by 18
- 2025SafeMobile: Chain-level Jailbreak Detection and Automated Evaluation for Multimodal Mobile Agentscited by 15
- 2024Object Detectors in the Open Environment: Challenges, Solutions, and Outlookcited by 13
- 2025Bench2ADVLM: A Closed-Loop Benchmark for Vision-language Models in Autonomous Drivingcited by 8
- 2025RoboDriveVLM: A Novel Benchmark and Baseline towards Robust Vision-Language Models for Autonomous Drivingcited by 8
- 2025Adversarial Generation and Collaborative Evolution of Safety-Critical Scenarios for Autonomous Vehiclescited by 5
- 2025Universal Camouflage Attack on Vision-Language Models for Autonomous Drivingcited by 4
- 2025MetAdv: A Unified and Interactive Adversarial Testing Platform for Autonomous Drivingcited by 3
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Herding End-to-End Autonomous Driving via Neuro-Symbolic Safety Guards
- 2025VLM-SAFE: Vision-Language Model-Guided Safety-Aware Reinforcement Learning with World Models for Autonomous Driving
- 2025LA-RL: Language Action-guided Reinforcement Learning with Safety Guarantees for Autonomous Highway Driving
- 2023Empowering Autonomous Driving with Large Language Models: A Safety Perspective
- 2024AD-H: Language-guided Autonomous Driving with Hierarchical Agents
- 2025SanDRA: Safe Large-Language-Model-Based Decision Making for Automated Vehicles Using Reachability Analysis
- 2026SafeDriveVLA: Navigation-Conditioned World Model Dreaming for Conflict-Aware End-to-End Autonomous Driving
- 2023Towards Safe Autonomous Driving Policies using a Neuro-Symbolic Deep Reinforcement Learning Approach