mlsanityData to 7 Oct 2026

ForesightSafety-VLA: A Unified Diagnostic Safety Benchmark for Vision-Language-Action Models

arXiv 2606.27079 ↗First posted 25 Jun 2026cs.ROcited by 2

On the map: Vision-language-action models → Safety and adversarial attacks

Abstract (arXiv)

In embodied intelligence, safety is a prerequisite for reliable robot deployment in the physical world. Current vision-language-action (VLA) models continue to advance toward general-purpose task capability, yet their embodied safety limits remain poorly understood. To address this gap, we introduce ForesightSafety-VLA, a diagnostic benchmark that makes safety the primary evaluation target for VLA systems. We define a 13-category safety taxonomy covering physical interaction safety (Safe-Core), instruction-side safety (Safe-Lang), and perception-side safety (Safe-Vis), and evaluate policies under three controlled dimensions of variation -- scene structure, language command, and visual observation -- so that failure sources can be diagnosed rather than hidden in a single aggregate score. Beyond binary task success, ForesightSafety-VLA measures process-level risk through cumulative safety cost (CC) and risk exposure time (RET), together with a four-quadrant decomposition of safe/unsafe success and failure. We instantiate 66 safety-augmented base scenarios in RoboTwin across 5 embodiments and report results on representative VLA baselines. Across the evaluated baselines, even the strongest policy incurs non-trivial safety cost and unsafe nominal success, while structure and visual variation induce substantially stronger safety degradation than ordinary language variation. These results suggest that embodied safety is tightly coupled to perception, grounding, and control competence rather than being reducible to post-hoc safety filtering alone.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Safety and adversarial attacks, one of 13 directions in Vision-language-action models. Builds on 36 of its 40 references that are in the map. First posted on arXiv 25 Jun 2026.

Builds on

36 of 40 references.

References · 36

  1. 2023Visual Instruction Tuningcited by 7,792
  2. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
  3. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
  4. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,730
  5. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,297
  6. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
  7. 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,653
  8. 2016Concrete Problems in AI Safetycited by 1,582
  9. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,490
  10. 2024Octo: An Open-Source Generalist Robot Policycited by 1,427
  11. 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,338
  12. 2023Open X-Embodiment: Robotic Learning Datasets and RT-X Modelscited by 849
24 more
  1. 2024RDT-1B: a Diffusion Foundation Model for Bimanual Manipulationcited by 670
  2. 2019RLBench: The Robot Learning Benchmark & Learning Environmentcited by 669
  3. 2020SAPIEN: A SimulAted Part-based Interactive ENvironmentcited by 584
  4. 2023Prompt Injection attack against LLM-integrated Applicationscited by 568
  5. 2021CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Taskscited by 564
  6. 2017Adversarial Patchcited by 514
  7. 2024Evaluating Real-World Robot Manipulation Policies in Simulationcited by 484
  8. 2025RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulationcited by 477
  9. 2024Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperationcited by 377
  10. 2021Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learningcited by 337
  11. 2024TinyVLA: Towards Fast, Data-Efficient Vision-Language-Action Models for Robotic Manipulationcited by 331
  12. 2023ManiSkill2: A Unified Benchmark for Generalizable Manipulation Skillscited by 313
  13. 2016Control Barrier Function Based Quadratic Programs for Safety Critical Systemscited by 261
  14. 2025DexVLA: Vision-Language Model with Plug-In Diffusion Expert for General Robot Controlcited by 191
  15. 2025RoboTwin: Dual-Arm Robot Benchmark with Generative Digital Twinscited by 117
  16. 2022SafeBench: A Benchmarking Platform for Safety Evaluation of Autonomous Vehiclescited by 65
  17. 2021RobustNav: Towards Benchmarking Robustness in Embodied Navigationcited by 52
  18. 2021safe-control-gym: a Unified Benchmark Suite for Safe Learning-based Control and Reinforcement Learning in Roboticscited by 42
  19. 2025Generating Robot Constitutions & Benchmarks for Semantic Safetycited by 28
  20. 2025VLA-Arena: An Open-Source Framework for Benchmarking Vision-Language-Action Modelscited by 28
  21. 2024Generative Image as Action Modelscited by 18
  22. 2025Safety of Embodied Navigation: A Surveycited by 9
  23. 2024Ensuring Force Safety in Vision-Guided Robotic Manipulation via Implicit Tactile Calibrationcited by 4
  24. 2025Towards Safe Robot Foundation Modelscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026HazardArena: Evaluating Semantic Safety in Vision-Language-Action Models
  2. 2026SafeVLA-Bench: A Benchmark for the Success-Safety Gap in Vision-Language-Action Models
  3. 2026Vision-Language-Action Safety: Threats, Challenges, Evaluations, and Mechanisms
  4. 2025AGENTSAFE: Benchmarking the Safety of Embodied Agents on Hazardous Instructions
  5. 2026SafeRelBench: A Spatial-Relation-Aware Benchmark for Process-Level Safety in VLM-Driven Embodied Agents
  6. 2026SafeStage: Evaluating Safety Before, During, and After Vision-Language-Conditioned Robot Manipulation ↗
  7. 2026VLPSA: Vision-Language-Poisson-Safe Actions for Full-Body Safety of Learned Policies
  8. 2025SafeVLA: Towards Safety Alignment of Vision-Language-Action Model via Constrained Learning