Embodied Hazard Mitigation using Vision-Language Models for Autonomous Mobile Robots
On the map: Multimodal LLMs / vision-language models → Driving VLMs and open-vocabulary detection
Abstract (arXiv)
Autonomous robots operating in dynamic environments should identify and report anomalies. Embodying proactive mitigation improves safety and operational continuity. This paper presents a multimodal anomaly detection and mitigation system that integrates vision-language models and large language models to identify and report hazardous situations and conflicts in real-time. The proposed system enables robots to perceive, interpret, report, and if possible respond to urban and environmental anomalies through proactive detection mechanisms and automated mitigation actions. A key contribution in this paper is the integration of Hazardous and Conflict states into the robot's decision-making framework, where each anomaly type can trigger specific mitigation strategies. User studies (n = 30) demonstrated the effectiveness of the system in anomaly detection with 91.2% prediction accuracy and relatively low latency response times using edge-ai architecture.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Driving VLMs and open-vocabulary detection, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 10 of its 22 references that are in the map. First posted on arXiv 8 Sep 2025.
Builds on
10 of 22 references.
References · 10
- 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,733
- 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
- 2023VLFM: Vision-Language Frontier Maps for Zero-Shot Semantic Navigationcited by 205
- 2021Explainable AI for Robot Failures: Generating Explanations that Improve User Assistance in Fault Recoverycited by 32
- 2022Explainable Reinforcement Learning via Model Transformscited by 11
- 2022Evaluating Human-like Explanations for Robot Actions in Reinforcement Learning Scenarioscited by 5
- 2024LLM-Enhanced Path Planning: Safe and Efficient Autonomous Navigation with Instructional Inputscited by 5
- 2023A Surrogate Model Framework for Explainable Autonomous Behaviourcited by 4
- 2024Enhancing Robot Explanation Capabilities through Vision-Language Models: a Preliminary Study by Interpreting Visual Inputs for Improved Human-Robot Interactioncited by 4
- 2024TalkWithMachines: Enhancing Human-Robot Interaction for Interpretable Industrial Robotics Through Large/Vision Language Modelscited by 4
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Hazard or Anomaly? Evaluating VLMs for Understanding Dangers and Discrepancies
- 2025Language as Cost: Proactive Hazard Mapping using VLM for Robot Navigation
- 2023Semantic Anomaly Detection with Large Language Models
- 2026SafeGuard ASF: SR Agentic Humanoid Robot System for Autonomous Industrial Safety
- 2024Real-Time Anomaly Detection and Reactive Planning with Large Language Models
- 2026ForesightSafety-VLA: A Unified Diagnostic Safety Benchmark for Vision-Language-Action Models
- 2025The Safety Challenge of World Models for Embodied AI Agents: A Review
- 2025Evaluation of Large Language Models for Anomaly Detection in Autonomous Vehicles