mlsanityData to 7 Oct 2026

HomeSafe-Bench: Evaluating Vision-Language Models on Unsafe Action Detection for Embodied Agents in Household Scenarios

arXiv 2603.11975 ↗First posted 12 Mar 2026cs.CV cs.AI cs.CRcited by 1

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

also in 1 more
Abstract (arXiv)

The rapid evolution of embodied agents has accelerated the deployment of household robots in real-world environments. However, unlike structured industrial settings, household spaces introduce unpredictable safety risks, where system limitations such as perception latency and lack of common sense knowledge can lead to dangerous errors. Current safety evaluations, often restricted to static images, text, or general hazards, fail to adequately benchmark dynamic unsafe action detection in these specific contexts. To bridge this gap, we introduce HomeSafe-Bench, a challenging benchmark designed to evaluate Vision-Language Models (VLMs) on unsafe action detection in household scenarios. HomeSafe-Bench is contrusted via a hybrid pipeline combining physical simulation with advanced video generation and features 438 diverse cases across six functional areas with fine-grained multidimensional annotations. Beyond benchmarking, we propose Hierarchical Dual-Brain Guard for Household Safety (HD-Guard), a hierarchical streaming architecture for real-time safety monitoring. HD-Guard coordinates a lightweight FastBrain for continuous high-frequency screening with an asynchronous large-scale SlowBrain for deep multimodal reasoning, effectively balancing inference efficiency with detection accuracy. Evaluations demonstrate that HD-Guard achieves a superior trade-off between latency and performance, while our analysis identifies critical bottlenecks in current VLM-based safety detection.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 23 of its 39 references that are in the map. First posted on arXiv 12 Mar 2026.

Builds on

23 of 39 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025VLM-3R: Vision-Language Models Augmented with Instruction-Aligned 3D Reconstruction26 May

Other references · 22

  1. 2024MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategiescited by 694
  2. 2022ProgPrompt: Generating Situated Robot Task Plans using Large Language Modelscited by 598
  3. 2023SayPlan: Grounding Large Language Models using 3D Scene Graphs for Scalable Robot Task Planningcited by 289
  4. 2023An Overview of Catastrophic AI Riskscited by 214
  5. 2025VITA-1.5: Towards GPT-4o Level Real-Time Vision and Speech Interactioncited by 191
  6. 2023SafetyBench: Evaluating the Safety of Large Language Modelscited by 189
  7. 2024Aligning Cyber Space with Physical World: A Comprehensive Survey on Embodied AIcited by 175
  8. 2024BEHAVIOR-1K: A Human-Centered, Embodied AI Benchmark with 1,000 Everyday Activities and Realistic Simulationcited by 140
  9. 2024SafeAgentBench: A Benchmark for Safe Task Planning of Embodied LLM Agentscited by 102
  10. 2025SafeVLA: Towards Safety Alignment of Vision-Language-Action Model via Constrained Learningcited by 69
  11. 2025IS-Bench: Evaluating Interactive Safety of VLM-Driven Embodied Agents in Daily Household Taskscited by 40
  12. 2025Treble Counterfactual VLMs: A Causal Approach to Hallucinationcited by 33
10 more
  1. 2025Towards Robust and Secure Embodied AI: A Survey on Vulnerabilities and Attackscited by 32
  2. 2025Generating Robot Constitutions & Benchmarks for Semantic Safetycited by 28
  3. 2025A Framework for Benchmarking and Aligning Task-Planning Safety in LLM-Based Embodied Agentscited by 27
  4. 2025Can AI Perceive Physical Danger and Intervene?cited by 12
  5. 2025Benchmarking the Robustness of Agentic Systems to Adversarially-Induced Harmscited by 10
  6. 2024Do Current Video LLMs Have Strong OCR Abilities? A Preliminary Studycited by 7
  7. 2024Open-Vocabulary Action Localization with Iterative Visual Promptingcited by 5
  8. 2026What Breaks Embodied AI Security:LLM Vulnerabilities, CPS Flaws,or Something Else?cited by 5
  9. 2025LiteVLM: A Low-Latency Vision-Language Model Inference Pipeline for Resource-Constrained Environmentscited by 4
  10. 2025Plan Verification for LLM-Based Embodied Task Completion Agentscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2026HomeGuard: VLM-based Embodied Safeguard for Identifying Contextual Risk in Household Task
  2. 2026SafeRelBench: A Spatial-Relation-Aware Benchmark for Process-Level Safety in VLM-Driven Embodied Agents
  3. 2025HomeSafeBench: Benchmarking Embodied Vision-Language Models in Free-Exploration Home Safety Inspection
  4. 2025RoboSafe: Safeguarding Embodied Agents via Executable Safety Logic
  5. 2026Self-Evolving Just-In-Time Memory for Proactive Embodied Safety
  6. 2026HazardArena: Evaluating Semantic Safety in Vision-Language-Action Models
  7. 2025AGENTSAFE: Benchmarking the Safety of Embodied Agents on Hazardous Instructions
  8. 2026SafeVLA-Bench: A Benchmark for the Success-Safety Gap in Vision-Language-Action Models