# CHAI: Command Hijacking against embodied AI

arXiv: https://arxiv.org/abs/2510.00181 · HTML: https://mlsanity.com/paper/2510.00181/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 65 references that are in the map.
- First posted on arXiv 30 Sep 2025.

## Builds on (most cited first, 30 of 35)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection](https://mlsanity.com/paper/2303.05499/) (2023)
- [Towards Evaluating the Robustness of Neural Networks](https://mlsanity.com/paper/1608.04644/) (2016)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling](https://mlsanity.com/paper/2412.05271/) (2024)
- [Jailbreaking Black Box Large Language Models in Twenty Queries](https://mlsanity.com/paper/2310.08419/) (2023)
- [DriveLM: Driving with Graph Visual Question Answering](https://mlsanity.com/paper/2312.14150/) (2023)
- [AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehicles](https://mlsanity.com/paper/1705.05065/) (2017)
- [DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Models](https://mlsanity.com/paper/2402.12289/) (2024)
- [DriveGPT4: Interpretable End-to-end Autonomous Driving via Large Language Model](https://mlsanity.com/paper/2310.01412/) (2023)
- [FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Prompts](https://mlsanity.com/paper/2311.05608/) (2023)
- [Visual Adversarial Examples Jailbreak Aligned Large Language Models](https://mlsanity.com/paper/2306.13213/) (2023)
- [OmniDrive: A Holistic Vision-Language Dataset for Autonomous Driving with Counterfactual Reasoning](https://mlsanity.com/paper/2504.04348/) (2025)
- [Dolphins: Multimodal Language Model for Driving](https://mlsanity.com/paper/2312.00438/) (2023)
- [Are VLMs Ready for Autonomous Driving? An Empirical Study from the Reliability, Data, and Metric Perspectives](https://mlsanity.com/paper/2501.04003/) (2025)
- [Automatic and Universal Prompt Injection Attacks against Large Language Models](https://mlsanity.com/paper/2403.04957/) (2024)
- [Scenic: A Language for Scenario Specification and Scene Generation](https://mlsanity.com/paper/1809.09310/) (2018)
- [Visual-RolePlay: Universal Jailbreak Attack on MultiModal Large Language Models via Role-playing Image Character](https://mlsanity.com/paper/2405.20773/) (2024)
- [Physical Adversarial Attack meets Computer Vision: A Decade Survey](https://mlsanity.com/paper/2209.15179/) (2022)
- [SafeGen: Mitigating Sexually Explicit Content Generation in Text-to-Image Models](https://mlsanity.com/paper/2404.06666/) (2024)
- [Dirty Road Can Attack: Security of Deep Learning based Automated Lane Centering under Physical-World Attack](https://mlsanity.com/paper/2009.06701/) (2020)
- [Drive Anywhere: Generalizable End-to-end Autonomous Driving with Multi-modal Foundation Models](https://mlsanity.com/paper/2310.17642/) (2023)
- [UAVs Meet LLMs: Overviews and Perspectives Toward Agentic Low-Altitude Mobility](https://mlsanity.com/paper/2501.02341/) (2025)
- [Vision-LLMs Can Fool Themselves with Self-Generated Typographic Attacks](https://mlsanity.com/paper/2402.00626/) (2024)
- [Adversarial Attacks on Robotic Vision Language Action Models](https://mlsanity.com/paper/2506.03350/) (2025)
- [ToViLaG: Your Visual-Language Generative Model is Also An Evildoer](https://mlsanity.com/paper/2312.11523/) (2023)
- [Stop Reasoning! When Multimodal LLM with Chain-of-Thought Reasoning Meets Adversarial Image](https://mlsanity.com/paper/2402.14899/) (2024)
- [Unveiling Typographic Deceptions: Insights of the Typographic Vulnerability in Large Vision-Language Model](https://mlsanity.com/paper/2402.19150/) (2024)

## Built on by (most cited first, 11 of 11)

- [Safety in Embodied AI: A Survey of Risks, Attacks, and Defenses](https://mlsanity.com/paper/2605.02900/) (2026)
- [Security of World-Model-Based Embodied AI: A Lifecycle of Threats, Defenses, and Evaluation](https://mlsanity.com/paper/2607.28226/) (2026)
- [Hijacking Robots with a Piece of Paper: A Systematic Study of Physical Prompt Injection in VLM-Controlled Robots](https://mlsanity.com/paper/2608.05715/) (2026)
- [An LLM-Agnostic, MAVLink-Based Drone Command and Control Interface and Agentic Harness Using the Model Context Protocol](https://mlsanity.com/paper/2601.15486/) (2026)
- [If you're waiting for a sign... that might not be it! Mitigating Trust Boundary Confusion from Visual Injections on Vision-Language Agentic Systems](https://mlsanity.com/paper/2604.19844/) (2026)
- [RIPA: Sensory-Vector Prompt Injection Attacks on LLM-Controlled ROS 2 Robots](https://mlsanity.com/paper/2606.28649/) (2026)
- [Not What You Asked For: Typographic Attacks in Household Robot Manipulation](https://mlsanity.com/paper/2605.18593/) (2026)
- [Overthink-Triggered Slowdown Attacks on LVLM-Based Robotic Systems](https://mlsanity.com/paper/2607.01518/) (2026)
- [Devil in the Lens: Analyzing and Defending Physical Prompt Injection Against Vision-Language Models on Wearable Devices](https://mlsanity.com/paper/2607.10269/) (2026)
- [Breaking Planner Integrity Boundary: Enviroment State-Text Injection Attack on LLM-Driven Embodied Agents](https://mlsanity.com/paper/2608.16806/) (2026)
- [Security and Privacy in Large-Model-Driven Embodied Agents: Attacks, Defenses, and Future Directions](https://mlsanity.com/paper/2609.27847/) (2026)

## Abstract (arXiv)

Embodied Artificial Intelligence (AI) promises to handle edge cases in robotic vehicle systems where data is scarce by using common-sense reasoning grounded in perception and action to generalize beyond training distributions and adapt to novel real-world situations. These capabilities, however, also create new security risks. In this paper, we introduce CHAI (Command Hijacking against embodied AI), a physical environment indirect prompt injection attack that exploits the multimodal language interpretation abilities of AI models. CHAI embeds deceptive natural language instructions, such as misleading signs, in visual input, systematically searches the token space, builds a dictionary of prompts, and guides an attacker model to generate Visual Attack Prompts. We evaluate CHAI on four LVLM agents: drone emergency landing, autonomous driving, aerial object tracking, and on a real robotic vehicle. Our experiments show that CHAI consistently outperforms state-of-the-art attacks. By exploiting the semantic and multimodal reasoning strengths of next-generation embodied AI systems, CHAI underscores the urgent need for defenses that extend beyond traditional adversarial robustness.