ForesightSafety-VLA: A Unified Diagnostic Safety Benchmark for Vision-Language-Action Models
On the map: Vision-language-action models → Safety and adversarial attacks
Abstract (arXiv)
In embodied intelligence, safety is a prerequisite for reliable robot deployment in the physical world. Current vision-language-action (VLA) models continue to advance toward general-purpose task capability, yet their embodied safety limits remain poorly understood. To address this gap, we introduce ForesightSafety-VLA, a diagnostic benchmark that makes safety the primary evaluation target for VLA systems. We define a 13-category safety taxonomy covering physical interaction safety (Safe-Core), instruction-side safety (Safe-Lang), and perception-side safety (Safe-Vis), and evaluate policies under three controlled dimensions of variation -- scene structure, language command, and visual observation -- so that failure sources can be diagnosed rather than hidden in a single aggregate score. Beyond binary task success, ForesightSafety-VLA measures process-level risk through cumulative safety cost (CC) and risk exposure time (RET), together with a four-quadrant decomposition of safe/unsafe success and failure. We instantiate 66 safety-augmented base scenarios in RoboTwin across 5 embodiments and report results on representative VLA baselines. Across the evaluated baselines, even the strongest policy incurs non-trivial safety cost and unsafe nominal success, while structure and visual variation induce substantially stronger safety degradation than ordinary language variation. These results suggest that embodied safety is tightly coupled to perception, grounding, and control competence rather than being reducible to post-hoc safety filtering alone.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Safety and adversarial attacks, one of 13 directions in Vision-language-action models. Builds on 36 of its 40 references that are in the map. First posted on arXiv 25 Jun 2026.
Builds on
36 of 40 references.
References · 36
- 2023Visual Instruction Tuningcited by 7,792
- 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
- 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
- 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,730
- 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,297
- 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
- 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,653
- 2016Concrete Problems in AI Safetycited by 1,582
- 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,490
- 2024Octo: An Open-Source Generalist Robot Policycited by 1,427
- 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,338
- 2023Open X-Embodiment: Robotic Learning Datasets and RT-X Modelscited by 849
24 more
- 2024RDT-1B: a Diffusion Foundation Model for Bimanual Manipulationcited by 670
- 2019RLBench: The Robot Learning Benchmark & Learning Environmentcited by 669
- 2020SAPIEN: A SimulAted Part-based Interactive ENvironmentcited by 584
- 2023Prompt Injection attack against LLM-integrated Applicationscited by 568
- 2021CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Taskscited by 564
- 2017Adversarial Patchcited by 514
- 2024Evaluating Real-World Robot Manipulation Policies in Simulationcited by 484
- 2025RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulationcited by 477
- 2024Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperationcited by 377
- 2021Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learningcited by 337
- 2024TinyVLA: Towards Fast, Data-Efficient Vision-Language-Action Models for Robotic Manipulationcited by 331
- 2023ManiSkill2: A Unified Benchmark for Generalizable Manipulation Skillscited by 313
- 2016Control Barrier Function Based Quadratic Programs for Safety Critical Systemscited by 261
- 2025DexVLA: Vision-Language Model with Plug-In Diffusion Expert for General Robot Controlcited by 191
- 2025RoboTwin: Dual-Arm Robot Benchmark with Generative Digital Twinscited by 117
- 2022SafeBench: A Benchmarking Platform for Safety Evaluation of Autonomous Vehiclescited by 65
- 2021RobustNav: Towards Benchmarking Robustness in Embodied Navigationcited by 52
- 2021safe-control-gym: a Unified Benchmark Suite for Safe Learning-based Control and Reinforcement Learning in Roboticscited by 42
- 2025Generating Robot Constitutions & Benchmarks for Semantic Safetycited by 28
- 2025VLA-Arena: An Open-Source Framework for Benchmarking Vision-Language-Action Modelscited by 28
- 2024Generative Image as Action Modelscited by 18
- 2025Safety of Embodied Navigation: A Surveycited by 9
- 2024Ensuring Force Safety in Vision-Guided Robotic Manipulation via Implicit Tactile Calibrationcited by 4
- 2025Towards Safe Robot Foundation Modelscited by 1
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2026HazardArena: Evaluating Semantic Safety in Vision-Language-Action Models
- 2026SafeVLA-Bench: A Benchmark for the Success-Safety Gap in Vision-Language-Action Models
- 2026Vision-Language-Action Safety: Threats, Challenges, Evaluations, and Mechanisms
- 2025AGENTSAFE: Benchmarking the Safety of Embodied Agents on Hazardous Instructions
- 2026SafeRelBench: A Spatial-Relation-Aware Benchmark for Process-Level Safety in VLM-Driven Embodied Agents
- 2026SafeStage: Evaluating Safety Before, During, and After Vision-Language-Conditioned Robot Manipulation ↗
- 2026VLPSA: Vision-Language-Poisson-Safe Actions for Full-Body Safety of Learned Policies
- 2025SafeVLA: Towards Safety Alignment of Vision-Language-Action Model via Constrained Learning