mlsanityData to 7 Oct 2026

ANNIE: Be Careful of Your Robots

arXiv 2509.03383 ↗First posted 3 Sep 2025cs.AI cs.ROcited by 2

On the map: Vision-language-action models → Safety and adversarial attacks

Abstract (arXiv)

The integration of vision-language-action (VLA) models into embodied AI (EAI) robots is rapidly advancing their ability to perform complex, long-horizon tasks in humancentric environments. However, EAI systems introduce critical security risks: a compromised VLA model can directly translate adversarial perturbations on sensory input into unsafe physical actions. Traditional safety definitions and methodologies from the machine learning community are no longer sufficient. EAI systems raise new questions, such as what constitutes safety, how to measure it, and how to design effective attack and defense mechanisms in physically grounded, interactive settings. In this work, we present the first systematic study of adversarial safety attacks on embodied AI systems, grounded in ISO standards for human-robot interactions. We (1) formalize a principled taxonomy of safety violations (critical, dangerous, risky) based on physical constraints such as separation distance, velocity, and collision boundaries; (2) introduce ANNIEBench, a benchmark of nine safety-critical scenarios with 2,400 video-action sequences for evaluating embodied safety; and (3) ANNIE-Attack, a task-aware adversarial framework with an attack leader model that decomposes long-horizon goals into frame-level perturbations. Our evaluation across representative EAI models shows attack success rates exceeding 50% across all safety categories. We further demonstrate sparse and adaptive attack strategies and validate the real-world impact through physical robot experiments. These results expose a previously underexplored but highly consequential attack surface in embodied AI systems, highlighting the urgent need for security-driven defenses in the physical AI era. Code is available at https://github.com/RLCLab/Annie.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Safety and adversarial attacks, one of 13 directions in Vision-language-action models. Builds on 62 of its 88 references that are in the map. First posted on arXiv 3 Sep 2025.

Builds on

62 of 88 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025BadVLA: Towards Backdoor Attacks on Vision-Language-Action Models via Objective-Decoupled Optimization22 May
  2. 2024Exploring the Adversarial Vulnerabilities of Vision-Language-Action Models in Robotics18 Nov
  3. 2024Manipulation Facing Threats: Evaluating Physical Vulnerabilities in End-to-End Vision Language Action Models20 Sep

Other references · 59

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  3. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  4. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
  5. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,976
  6. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
  7. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,730
  8. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,297
  9. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
  10. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
  11. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,490
  12. 2024Octo: An Open-Source Generalist Robot Policycited by 1,427
38 more
  1. 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,338
  2. 2019Habitat: A Platform for Embodied AI Researchcited by 1,163
  3. 2017AI2-THOR: An Interactive 3D Environment for Visual AIcited by 850
  4. 2023VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Modelscited by 733
  5. 2019RLBench: The Robot Learning Benchmark & Learning Environmentcited by 669
  6. 2020SAPIEN: A SimulAted Part-based Interactive ENvironmentcited by 584
  7. 2021CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Taskscited by 564
  8. 2016Delving into Transferable Adversarial Examples and Black-box Attackscited by 530
  9. 2017Countering Adversarial Images using Input Transformationscited by 517
  10. 2017Feature Squeezing: Detecting Adversarial Examples in Deep Neural Networkscited by 484
  11. 2022LLM-Planner: Few-Shot Grounded Planning for Embodied Agents with Large Language Modelscited by 459
  12. 2025CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Modelscited by 440
  13. 2017Mitigating Adversarial Effects Through Randomizationcited by 382
  14. 2024Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperationcited by 377
  15. 2017Adversarial Examples: Attacks and Defenses for Deep Learningcited by 350
  16. 2024Robotic Control via Embodied Chain-of-Thought Reasoningcited by 340
  17. 2024GR-2: A Generative Video-Language-Action Model with Web-Scale Knowledge for Robot Manipulationcited by 290
  18. 2023SayPlan: Grounding Large Language Models using 3D Scene Graphs for Scalable Robot Task Planningcited by 289
  19. 2019Adversarial Examples Improve Image Recognitioncited by 283
  20. 2023EmbodiedGPT: Vision-Language Pre-Training via Embodied Chain of Thoughtcited by 276
  21. 2017MagNet: a Two-Pronged Defense against Adversarial Examplescited by 268
  22. 2020ThreeDWorld: A Platform for Interactive Multi-Modal Physical Simulationcited by 249
  23. 2023Building Cooperative Embodied Agents Modularly with Large Language Modelscited by 231
  24. 2023RoCo: Dialectic Multi-Robot Collaboration with Large Language Modelscited by 171
  25. 2024VLABench: A Large-Scale Benchmark for Language-Conditioned Robotics Manipulation with Long-Horizon Reasoning Taskscited by 171
  26. 2020Watch-And-Help: A Challenge for Social Perception and Human-AI Collaborationcited by 131
  27. 2024SafeAgentBench: A Benchmark for Safe Task Planning of Embodied LLM Agentscited by 102
  28. 2023Embodied Task Planning with Large Language Modelscited by 84
  29. 2024BAKU: An Efficient Transformer for Multi-Task Policy Learningcited by 80
  30. 2024A Survey on Robotics with Foundation Models: toward Embodied AIcited by 53
  31. 2024COMBO: Compositional World Models for Embodied Multi-Agent Cooperationcited by 47
  32. 2024KARMA: Augmenting Embodied AI Agents with Long-and-short Term Memory Systemscited by 38
  33. 2025Red Teaming the Mind of the Machine: A Systematic Evaluation of Prompt Injection and Jailbreak Vulnerabilities in LLMscited by 34
  34. 2025Towards Robust and Secure Embodied AI: A Survey on Vulnerabilities and Attackscited by 32
  35. 2024Exploring the Robustness of Decision-Level Through Adversarial Attacks on LLM-Based Embodied Modelscited by 31
  36. 2025ManipLVM-R1: Reinforcement Learning for Reasoning in Embodied Manipulation with Large Vision-Language Modelscited by 26
  37. 2020Adversarial Machine Learning in Wireless Communications using RF Data: A Reviewcited by 25
  38. 2024LLM-Personalize: Aligning LLM Planners with Human Preferences via Reinforced Self-Training for Housekeeping Robotscited by 18

50 most cited of 59 All 59 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026Vision-Language-Action Safety: Threats, Challenges, Evaluations, and Mechanisms
  2. 2026Safety in Embodied AI: A Survey of Risks, Attacks, and Defenses
  3. 2025AGENTSAFE: Benchmarking the Safety of Embodied Agents on Hazardous Instructions
  4. 2026RoboJailBench: Benchmarking Adversarial Attacks and Defenses in Embodied Robotic Agents
  5. 2026HazardArena: Evaluating Semantic Safety in Vision-Language-Action Models
  6. 2025Phantom Menace: Exploring and Enhancing the Robustness of VLA Models Against Physical Sensor Attacks
  7. 2026GuardianBench: A Same-Scene Instruction-Contrastive Benchmark for Latent Contextual Risk in Embodied AI
  8. 2026ForesightSafety-VLA: A Unified Diagnostic Safety Benchmark for Vision-Language-Action Models