mlsanityData to 7 Oct 2026

SafeRelBench: A Spatial-Relation-Aware Benchmark for Process-Level Safety in VLM-Driven Embodied Agents

arXiv 2607.14543 ↗First posted 16 Jul 2026cs.RO cs.AIcited by 1

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

also in 1 more
Abstract (arXiv)

Vision-language models (VLMs) are increasingly used as the reasoning backbone of embodied agents, enabling robots to interpret visual scenes, follow language instructions, and plan multi-step actions. In household environments, however, safety depends not only on recognizing objects, but also on how actions change the physical scene over time. Existing embodied safety evaluations largely focus on static risk recognition, unsafe instruction refusal, or final-state task completion. As a result, process-level safety failures induced by spatial relations such as support, containment, and proximity remain insufficiently studied. To address this gap, we introduce SAFERELBENCH, a spatial-relation-aware safety benchmark with 507 executable evaluation samples, including 248 spatial-relation samples and 259 non-spatial control samples. Using SAFERELBENCH to evaluate seven open- and closed-source VLM-driven embodied agents, we find a substantial gap between task success and process-level safety compliance: models often complete the requested task while violating process-level safety constraints. Unlike prior benchmarks, SAFERELBENCH explicitly tests whether agents satisfy safety conditions before risk-prone actions, making spatial relations a core dimension in embodied safety assessment. More broadly, our results show that safe embodied intelligence requires not only stronger perception and planning, but also reliable reasoning about how object relations shape risk during interaction.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 23 of its 32 references that are in the map. First posted on arXiv 16 Jul 2026.

Builds on

23 of 32 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024RoboSpatial: Teaching Spatial Understanding to 2D and 3D Vision-Language Models for Robotics25 Nov
  2. 2023Physically Grounded Vision-Language Models for Robotic Manipulation5 Sep
  3. 2023EmbodiedGPT: Vision-Language Pre-Training via Embodied Chain of Thought24 May

Other references · 20

  1. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
  2. 2019Habitat: A Platform for Embodied AI Researchcited by 1,163
  3. 2022Inner Monologue: Embodied Reasoning through Planning with Language Modelscited by 1,057
  4. 2017AI2-THOR: An Interactive 3D Environment for Visual AIcited by 850
  5. 2019ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Taskscited by 773
  6. 2018VirtualHome: Simulating Household Activities via Programscited by 411
  7. 2022VIMA: General Robot Manipulation with Multimodal Promptscited by 403
  8. 2019REVERIE: Remote Embodied Visual Referring Expression in Real Indoor Environmentscited by 330
  9. 2021TEACh: Task-driven Embodied Agents that Chatcited by 194
  10. 2024Aligning Cyber Space with Physical World: A Comprehensive Survey on Embodied AIcited by 175
  11. 2024SafeAgentBench: A Benchmark for Safe Task Planning of Embodied LLM Agentscited by 102
  12. 2022Housekeep: Tidying Virtual Households using Commonsense Reasoningcited by 57
8 more
  1. 2024A Survey on Robotics with Foundation Models: toward Embodied AIcited by 53
  2. 2017Visual Semantic Planning using Deep Successor Representationscited by 45
  3. 2025IS-Bench: Evaluating Interactive Safety of VLM-Driven Embodied Agents in Daily Household Taskscited by 40
  4. 2025A Framework for Benchmarking and Aligning Task-Planning Safety in LLM-Based Embodied Agentscited by 27
  5. 2025Subtle Risks, Critical Failures: A Framework for Diagnosing Physical Safety of LLMs for Embodied Decision Makingcited by 19
  6. 2025Advancing Embodied Agent Security: From Safety Benchmarks to Input Moderationcited by 12
  7. 2023Grounding Object Relations in Language-Conditioned Robotic Manipulation with Semantic-Spatial Reasoningcited by 4
  8. 2026BeSafe-Bench: Unveiling Behavioral Safety Risks of Situated Agents in Functional Environmentscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2026ForesightSafety-VLA: A Unified Diagnostic Safety Benchmark for Vision-Language-Action Models
  2. 2026HomeGuard: VLM-based Embodied Safeguard for Identifying Contextual Risk in Household Task
  3. 2026HazardArena: Evaluating Semantic Safety in Vision-Language-Action Models
  4. 2025AGENTSAFE: Benchmarking the Safety of Embodied Agents on Hazardous Instructions
  5. 2026SafeVLA-Bench: A Benchmark for the Success-Safety Gap in Vision-Language-Action Models
  6. 2026GuardianBench: A Same-Scene Instruction-Contrastive Benchmark for Latent Contextual Risk in Embodied AI
  7. 2026Contextual Safety Reasoning and Grounding for Open-World Robots
  8. 2025SafeMind: Benchmarking and Mitigating Safety Risks in Embodied LLM Agents