mlsanityData to 14 Oct 2026

HomeSafeBench: Benchmarking Embodied Vision-Language Models in Free-Exploration Home Safety Inspection

arXiv 2509.23690 ↗First posted 28 Sep 2025cs.CV cs.CLcited by 5

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

Abstract (arXiv)

Safety hazards in the home are a leading cause of preventable domestic injuries, motivating an automated inspector that actively explores a home and reports hazards before they cause harm. We introduce HomeSafeBench, the first benchmark for free-exploration home safety inspection with egocentric visual feedback, in which an embodied agent navigates a fully interactive 3D home, adjusts its viewpoint, and reports hazards purely from rendered first-person views. Built on the VirtualHome simulator, it covers five categories of common household hazards and comprises 1,000 human-validated inspection tasks. Evaluating a broad range of state-of-the-art Vision-Language Models (VLMs) reveals a large gap, where the best model reaches only about 34.7% F1, far below the 98.0% of a human inspector. Moreover, precision far exceeds recall across models, revealing a systematic tendency to under-report hazards that reflects a shared deficiency in risk recognition. To close this gap at low cost, we propose CueBack, an offline data-construction method that exploits the clue-precedes-confirmation structure of inspection, backtracking a privileged trajectory to the earliest frame where a hazard cue becomes visible and rewriting it into executable supervision. Fine-tuning a 4B-size VLM on CueBack-constructed data raises the average F1 from 18.7% to 45.3% on an out-of-distribution test set, surpassing the strongest closed-source model performance 34.7%. The benchmark, training dataset, and code are available at https://github.com/BITHLP/HomeSafeBench.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 19 of its 45 references that are in the map. First posted on arXiv 28 Sep 2025.

Builds on

19 of 45 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025AGENTSAFE: Benchmarking the Safety of Embodied Agents on Hazardous Instructions17 Jun

Other references · 18

  1. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  2. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,027
  3. 2018VirtualHome: Simulating Household Activities via Programscited by 415
  4. 2017Embodied Question Answeringcited by 336
  5. 2021A Survey of Embodied AI: From Simulators to Research Taskscited by 261
  6. 2020ObjectNav Revisited: On Evaluation of Embodied Agents Navigating to Objectscited by 244
  7. 2019Learning Exploration Policies for Navigationcited by 116
  8. 2024SafeAgentBench: A Benchmark for Safe Task Planning of Embodied LLM Agentscited by 116
  9. 2020Auxiliary Tasks Speed Up Learning PointGoal Navigationcited by 54
  10. 2025IS-Bench: Evaluating Interactive Safety of VLM-Driven Embodied Agents in Daily Household Taskscited by 41
  11. 2024Exploring the Robustness of Decision-Level Through Adversarial Attacks on LLM-Based Embodied Modelscited by 33
  12. 2024HAZARD Challenge: Embodied Decision Making in Dynamically Changing Environmentscited by 32
6 more
  1. 2025A Framework for Benchmarking and Aligning Task-Planning Safety in LLM-Based Embodied Agentscited by 30
  2. 2025AVD2: Accident Video Diffusion for Accident Video Descriptioncited by 20
  3. 2025Subtle Risks, Critical Failures: A Framework for Diagnosing Physical Safety of LLMs for Embodied Decision Makingcited by 20
  4. 2024SafeEmbodAI: a Safety Framework for Mobile Robots in Embodied AI Systemscited by 13
  5. 2024"Don't forget to put the milk back!" Dataset for Enabling Embodied Agents to Detect Anomalous Situationscited by 9
  6. 2024Coherence-Driven Multimodal Safety Dialogue with Active Learning for Embodied Agentscited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 5.

5 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026HomeSafe-Bench: Evaluating Vision-Language Models on Unsafe Action Detection for Embodied Agents in Household Scenarios
  2. 2026EgoSafetyBench: A Diagnostic Egocentric Video Benchmark for Evaluating Embodied VLMs as Runtime Safety Guards
  3. 2026SafeRelBench: A Spatial-Relation-Aware Benchmark for Process-Level Safety in VLM-Driven Embodied Agents
  4. 2026Towards Generalizable Visually Grounded Exploration of Household Devices
  5. 2026SafetyALFRED: Evaluating Safety-Conscious Planning of Multimodal Large Language Models
  6. 2026ForesightSafety-VLA: A Unified Diagnostic Safety Benchmark for Vision-Language-Action Models
  7. 2026GuardianBench: A Same-Scene Instruction-Contrastive Benchmark for Latent Contextual Risk in Embodied AI
  8. 2026Self-Evolving Just-In-Time Memory for Proactive Embodied Safety