ANNIE: Be Careful of Your Robots
On the map: Vision-language-action models → Safety and adversarial attacks
Abstract (arXiv)
The integration of vision-language-action (VLA) models into embodied AI (EAI) robots is rapidly advancing their ability to perform complex, long-horizon tasks in humancentric environments. However, EAI systems introduce critical security risks: a compromised VLA model can directly translate adversarial perturbations on sensory input into unsafe physical actions. Traditional safety definitions and methodologies from the machine learning community are no longer sufficient. EAI systems raise new questions, such as what constitutes safety, how to measure it, and how to design effective attack and defense mechanisms in physically grounded, interactive settings. In this work, we present the first systematic study of adversarial safety attacks on embodied AI systems, grounded in ISO standards for human-robot interactions. We (1) formalize a principled taxonomy of safety violations (critical, dangerous, risky) based on physical constraints such as separation distance, velocity, and collision boundaries; (2) introduce ANNIEBench, a benchmark of nine safety-critical scenarios with 2,400 video-action sequences for evaluating embodied safety; and (3) ANNIE-Attack, a task-aware adversarial framework with an attack leader model that decomposes long-horizon goals into frame-level perturbations. Our evaluation across representative EAI models shows attack success rates exceeding 50% across all safety categories. We further demonstrate sparse and adaptive attack strategies and validate the real-world impact through physical robot experiments. These results expose a previously underexplored but highly consequential attack surface in embodied AI systems, highlighting the urgent need for security-driven defenses in the physical AI era. Code is available at https://github.com/RLCLab/Annie.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Safety and adversarial attacks, one of 13 directions in Vision-language-action models. Builds on 62 of its 88 references that are in the map. First posted on arXiv 3 Sep 2025.
Builds on
62 of 88 references.
Closest ancestors · 3
Same direction, newest first
- 2025BadVLA: Towards Backdoor Attacks on Vision-Language-Action Models via Objective-Decoupled Optimization22 May
- 2024Exploring the Adversarial Vulnerabilities of Vision-Language-Action Models in Robotics18 Nov
- 2024Manipulation Facing Threats: Evaluating Physical Vulnerabilities in End-to-End Vision Language Action Models20 Sep
Other references · 59
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2014Explaining and Harnessing Adversarial Examplescited by 7,287
- 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
- 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
- 2016Towards Evaluating the Robustness of Neural Networkscited by 2,976
- 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
- 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,730
- 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,297
- 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
- 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
- 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,490
- 2024Octo: An Open-Source Generalist Robot Policycited by 1,427
38 more
- 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,338
- 2019Habitat: A Platform for Embodied AI Researchcited by 1,163
- 2017AI2-THOR: An Interactive 3D Environment for Visual AIcited by 850
- 2023VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Modelscited by 733
- 2019RLBench: The Robot Learning Benchmark & Learning Environmentcited by 669
- 2020SAPIEN: A SimulAted Part-based Interactive ENvironmentcited by 584
- 2021CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Taskscited by 564
- 2016Delving into Transferable Adversarial Examples and Black-box Attackscited by 530
- 2017Countering Adversarial Images using Input Transformationscited by 517
- 2017Feature Squeezing: Detecting Adversarial Examples in Deep Neural Networkscited by 484
- 2022LLM-Planner: Few-Shot Grounded Planning for Embodied Agents with Large Language Modelscited by 459
- 2025CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Modelscited by 440
- 2017Mitigating Adversarial Effects Through Randomizationcited by 382
- 2024Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperationcited by 377
- 2017Adversarial Examples: Attacks and Defenses for Deep Learningcited by 350
- 2024Robotic Control via Embodied Chain-of-Thought Reasoningcited by 340
- 2024GR-2: A Generative Video-Language-Action Model with Web-Scale Knowledge for Robot Manipulationcited by 290
- 2023SayPlan: Grounding Large Language Models using 3D Scene Graphs for Scalable Robot Task Planningcited by 289
- 2019Adversarial Examples Improve Image Recognitioncited by 283
- 2023EmbodiedGPT: Vision-Language Pre-Training via Embodied Chain of Thoughtcited by 276
- 2017MagNet: a Two-Pronged Defense against Adversarial Examplescited by 268
- 2020ThreeDWorld: A Platform for Interactive Multi-Modal Physical Simulationcited by 249
- 2023Building Cooperative Embodied Agents Modularly with Large Language Modelscited by 231
- 2023RoCo: Dialectic Multi-Robot Collaboration with Large Language Modelscited by 171
- 2024VLABench: A Large-Scale Benchmark for Language-Conditioned Robotics Manipulation with Long-Horizon Reasoning Taskscited by 171
- 2020Watch-And-Help: A Challenge for Social Perception and Human-AI Collaborationcited by 131
- 2024SafeAgentBench: A Benchmark for Safe Task Planning of Embodied LLM Agentscited by 102
- 2023Embodied Task Planning with Large Language Modelscited by 84
- 2024BAKU: An Efficient Transformer for Multi-Task Policy Learningcited by 80
- 2024A Survey on Robotics with Foundation Models: toward Embodied AIcited by 53
- 2024COMBO: Compositional World Models for Embodied Multi-Agent Cooperationcited by 47
- 2024KARMA: Augmenting Embodied AI Agents with Long-and-short Term Memory Systemscited by 38
- 2025Red Teaming the Mind of the Machine: A Systematic Evaluation of Prompt Injection and Jailbreak Vulnerabilities in LLMscited by 34
- 2025Towards Robust and Secure Embodied AI: A Survey on Vulnerabilities and Attackscited by 32
- 2024Exploring the Robustness of Decision-Level Through Adversarial Attacks on LLM-Based Embodied Modelscited by 31
- 2025ManipLVM-R1: Reinforcement Learning for Reasoning in Embodied Manipulation with Large Vision-Language Modelscited by 26
- 2020Adversarial Machine Learning in Wireless Communications using RF Data: A Reviewcited by 25
- 2024LLM-Personalize: Aligning LLM Planners with Human Preferences via Reinforced Self-Training for Housekeeping Robotscited by 18
50 most cited of 59 All 59 (JSON)
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2026Vision-Language-Action Safety: Threats, Challenges, Evaluations, and Mechanisms
- 2026Safety in Embodied AI: A Survey of Risks, Attacks, and Defenses
- 2025AGENTSAFE: Benchmarking the Safety of Embodied Agents on Hazardous Instructions
- 2026RoboJailBench: Benchmarking Adversarial Attacks and Defenses in Embodied Robotic Agents
- 2026HazardArena: Evaluating Semantic Safety in Vision-Language-Action Models
- 2025Phantom Menace: Exploring and Enhancing the Robustness of VLA Models Against Physical Sensor Attacks
- 2026GuardianBench: A Same-Scene Instruction-Contrastive Benchmark for Latent Contextual Risk in Embodied AI
- 2026ForesightSafety-VLA: A Unified Diagnostic Safety Benchmark for Vision-Language-Action Models