# Autonomous VR-Based Risk Detection for Situational Awareness in Dangerous Settings

arXiv: https://arxiv.org/abs/2607.16582 · HTML: https://mlsanity.com/paper/2607.16582/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 41 references that are in the map.
- First posted on arXiv 18 Jul 2026.

## Builds on (most cited first, 17 of 17)

- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [Vision-Language Models for Vision Tasks: A Survey](https://mlsanity.com/paper/2304.00685/) (2023)
- [Winoground: Probing Vision and Language Models for Visio-Linguistic Compositionality](https://mlsanity.com/paper/2204.03162/) (2022)
- [Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theory](https://mlsanity.com/paper/2310.17884/) (2023)
- [Power Hungry Processing: Watts Driving the Cost of AI Deployment?](https://mlsanity.com/paper/2311.16863/) (2023)
- [TinyLLaVA: A Framework of Small-scale Large Multimodal Models](https://mlsanity.com/paper/2402.14289/) (2024)
- [Understanding Large-Language Model (LLM)-powered Human-Robot Interaction](https://mlsanity.com/paper/2401.03217/) (2024)
- [CoPAL: Corrective Planning of Robot Actions with Large Language Models](https://mlsanity.com/paper/2310.07263/) (2023)
- [Do LLMs Understand Visual Anomalies? Uncovering LLM's Capabilities in Zero-shot Anomaly Detection](https://mlsanity.com/paper/2404.09654/) (2024)
- [The Dark Side of Dataset Scaling: Evaluating Racial Classification in Multimodal Models](https://mlsanity.com/paper/2405.04623/) (2024)
- [MSTS: A Multimodal Safety Test Suite for Vision-Language Models](https://mlsanity.com/paper/2501.10057/) (2025)
- [Common Inpainted Objects In-N-Out of Context](https://mlsanity.com/paper/2506.00721/) (2025)
- [Better Safe Than Sorry? Overreaction Problem of Vision Language Models in Visual Emergency Recognition](https://mlsanity.com/paper/2505.15367/) (2025)
- [Trustworthy Anomaly Detection: A Survey](https://mlsanity.com/paper/2202.07787/) (2022)
- [Multi-Sensory HMI for Human-Centric Industrial Digital Twins: A 6G Vision of Future Industry](https://mlsanity.com/paper/2111.10438/) (2021)
- [HRI Challenges Influencing Low Usage of Robotic Systems in Disaster Response and Rescue Operations](https://mlsanity.com/paper/2401.15760/) (2024)

## Abstract (arXiv)

In high-risk environments such as disaster response, situational awareness depends not only on detecting hazards but also on communicating them clearly to human operators. Vision Language Models (VLMs) have shown strong potential for scene understanding in safety-critical settings, yet their value as part of human-facing robotic systems remains underexplored. We present a VR-based Human Robot Interaction framework for studying how VLM-assisted robots can support situational awareness in simulated hazardous environments. In our system, a robot explores a virtual scene and queries a VLM to identify potential hazards and annotate user-facing points of interest. These annotations are presented to a human operator through an immersive VR interface. This framework enables controlled evaluation of both robotic hazard identification and the communication of safety-critical information to users. Results from our study indicate that the annotated VR interface was preferred over the unannotated baseline and that participants reported high clarity, usefulness, and comfort when interacting with the system. These findings suggest that combining VLM-based robotic perception with immersive visualization is a promising approach for supporting situational awareness in hazardous settings.