HomeSafeBench: Benchmarking Embodied Vision-Language Models in Free-Exploration Home Safety Inspection
On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models
Abstract (arXiv)
Safety hazards in the home are a leading cause of preventable domestic injuries, motivating an automated inspector that actively explores a home and reports hazards before they cause harm. We introduce HomeSafeBench, the first benchmark for free-exploration home safety inspection with egocentric visual feedback, in which an embodied agent navigates a fully interactive 3D home, adjusts its viewpoint, and reports hazards purely from rendered first-person views. Built on the VirtualHome simulator, it covers five categories of common household hazards and comprises 1,000 human-validated inspection tasks. Evaluating a broad range of state-of-the-art Vision-Language Models (VLMs) reveals a large gap, where the best model reaches only about 34.7% F1, far below the 98.0% of a human inspector. Moreover, precision far exceeds recall across models, revealing a systematic tendency to under-report hazards that reflects a shared deficiency in risk recognition. To close this gap at low cost, we propose CueBack, an offline data-construction method that exploits the clue-precedes-confirmation structure of inspection, backtracking a privileged trajectory to the earliest frame where a hazard cue becomes visible and rewriting it into executable supervision. Fine-tuning a 4B-size VLM on CueBack-constructed data raises the average F1 from 18.7% to 45.3% on an out-of-distribution test set, surpassing the strongest closed-source model performance 34.7%. The benchmark, training dataset, and code are available at https://github.com/BITHLP/HomeSafeBench.
Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 19 of its 45 references that are in the map. First posted on arXiv 28 Sep 2025.
Builds on
19 of 45 references.
Closest ancestors · 1
Same direction, newest first
Other references · 18
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
- 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,027
- 2018VirtualHome: Simulating Household Activities via Programscited by 415
- 2017Embodied Question Answeringcited by 336
- 2021A Survey of Embodied AI: From Simulators to Research Taskscited by 261
- 2020ObjectNav Revisited: On Evaluation of Embodied Agents Navigating to Objectscited by 244
- 2019Learning Exploration Policies for Navigationcited by 116
- 2024SafeAgentBench: A Benchmark for Safe Task Planning of Embodied LLM Agentscited by 116
- 2020Auxiliary Tasks Speed Up Learning PointGoal Navigationcited by 54
- 2025IS-Bench: Evaluating Interactive Safety of VLM-Driven Embodied Agents in Daily Household Taskscited by 41
- 2024Exploring the Robustness of Decision-Level Through Adversarial Attacks on LLM-Based Embodied Modelscited by 33
- 2024HAZARD Challenge: Embodied Decision Making in Dynamically Changing Environmentscited by 32
6 more
- 2025A Framework for Benchmarking and Aligning Task-Planning Safety in LLM-Based Embodied Agentscited by 30
- 2025AVD2: Accident Video Diffusion for Accident Video Descriptioncited by 20
- 2025Subtle Risks, Critical Failures: A Framework for Diagnosing Physical Safety of LLMs for Embodied Decision Makingcited by 20
- 2024SafeEmbodAI: a Safety Framework for Mobile Robots in Embodied AI Systemscited by 13
- 2024"Don't forget to put the milk back!" Dataset for Enabling Embodied Agents to Detect Anomalous Situationscited by 9
- 2024Coherence-Driven Multimodal Safety Dialogue with Active Learning for Embodied Agentscited by 8
Built on by
Citing papers per year: 2026: 5.
5 papers.
- 2026SpaMEM: Benchmarking Dynamic Spatial Reasoning via Perception-Memory Integration in Embodied Environmentscited by 6
- 2026HomeGuard: VLM-based Embodied Safeguard for Identifying Contextual Risk in Household Taskcited by 5
- 2026Safe Embodied AI for Long-horizon Tasks: A Cross-layer Analysis of Robotic Manipulationcited by 1
- 2026TSHA: A Benchmark for Visual Language Models in Trustworthy Safety Hazard Assessment Scenarioscited by 0
- 2026UniETP: Unifying Environments for Generalizable Embodied Task Planningcited by 0
- 2026UniETP: Unifying Environments for Generalizable Embodied Task Planning20 Jul 2026
- 2026Safe Embodied AI for Long-horizon Tasks: A Cross-layer Analysis of Robotic Manipulation4 Jun 2026
- 2026SpaMEM: Benchmarking Dynamic Spatial Reasoning via Perception-Memory Integration in Embodied Environments24 Apr 2026
- 2026TSHA: A Benchmark for Visual Language Models in Trustworthy Safety Hazard Assessment Scenarios31 Mar 2026
- 2026HomeGuard: VLM-based Embodied Safeguard for Identifying Contextual Risk in Household Task15 Mar 2026
Similar, not cited
Close in wording, no citation either way
- 2026HomeSafe-Bench: Evaluating Vision-Language Models on Unsafe Action Detection for Embodied Agents in Household Scenarios
- 2026EgoSafetyBench: A Diagnostic Egocentric Video Benchmark for Evaluating Embodied VLMs as Runtime Safety Guards
- 2026SafeRelBench: A Spatial-Relation-Aware Benchmark for Process-Level Safety in VLM-Driven Embodied Agents
- 2026Towards Generalizable Visually Grounded Exploration of Household Devices
- 2026SafetyALFRED: Evaluating Safety-Conscious Planning of Multimodal Large Language Models
- 2026ForesightSafety-VLA: A Unified Diagnostic Safety Benchmark for Vision-Language-Action Models
- 2026GuardianBench: A Same-Scene Instruction-Contrastive Benchmark for Latent Contextual Risk in Embodied AI
- 2026Self-Evolving Just-In-Time Memory for Proactive Embodied Safety