Breaking Planner Integrity Boundary: Enviroment State-Text Injection Attack on LLM-Driven Embodied Agents
On the map: LLM agents and tool use → Agent threats, guardrails and governance
Abstract (arXiv)
Large language model (LLM)-driven embodied agents rely on environment states to interpret scenes, generate high-level plans, and drive physical execution, making planner-visible state representations a critical security boundary. Existing attacks primarily manipulate user instructions, prompt contexts, model behavior, or perceptual inputs, while paying limited attention to whether environment-state text itself can serve as deceptive task evidence and propagate beyond planning to affect execution outcomes. Because embodied tasks are constrained by entity grounding, action preconditions, spatial relations, and environmental constraints, planning deviation alone does not guarantee adversarial execution. To address this gap, we investigate environment-state text as an independent attack surface and present the first closed-loop Environment State-Text Injection (ESTI) attack for LLM-driven embodied agents. Without modifying the original user instruction, model parameters, or executor, ESTI reformulates an adversarial objective as false state evidence compatible with the current environment and influences planning and execution through object properties, spatial relations, affordances, task-stage rules, and execution feedback. We further develop ESTI-Bench to evaluate attack propagation across the planning-to-execution closed loop and compare ESTI with Vanilla IPI, EIRAD, and BADROBOT across ProgPrompt/VirtualHome, VoxPoser/RLBench, and AI2-THOR/iTHOR. ESTI consistently outperforms existing baselines, improving planning-level and execution-level attack success rates by up to 89.32% and 43.69%, respectively. Further analysis shows that grounding, consistency, and executability jointly determine whether manipulated state evidence can propagate through the embodied closed loop and produce verifiable environmental changes.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 56 of its 62 references that are in the map. First posted on arXiv 17 Aug 2026.
Builds on
56 of 62 references.
Closest ancestors · 2
Same direction, newest first
- 2024BadRobot: Jailbreaking Embodied LLM Agents in the Physical World16 Jul
- 2024InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents5 Mar
Other references · 54
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
- 2021Evaluating Large Language Models Trained on Codecited by 8,320
- 2014Explaining and Harnessing Adversarial Examplescited by 7,364
- 2013Intriguing properties of neural networkscited by 5,155
- 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
- 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
- 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
- 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
- 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
- 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
38 more
- 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,348
- 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,191
- 2020Extracting Training Data from Large Language Modelscited by 2,189
- 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,071
- 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,947
- 2016Adversarial examples in the physical worldcited by 1,769
- 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,745
- 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,737
- 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,651
- 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,443
- 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,332
- 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agentscited by 1,251
- 2023Mind2Web: Towards a Generalist Agent for the Webcited by 1,242
- 2025GR00T N1: An Open Foundation Model for Generalist Humanoid Robotscited by 1,225
- 2022Code as Policies: Language Model Programs for Embodied Controlcited by 1,210
- 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,205
- 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,195
- 2019Habitat: A Platform for Embodied AI Researchcited by 1,191
- 2022Inner Monologue: Embodied Reasoning through Planning with Language Modelscited by 1,081
- 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 1,035
- 2022Red Teaming Language Models with Language Modelscited by 1,026
- 2024OpenHands: An Open Platform for AI Software Developers as Generalist Agentscited by 1,002
- 2017AI2-THOR: An Interactive 3D Environment for Visual AIcited by 869
- 2022Ignore Previous Prompt: Attack Techniques For Language Modelscited by 832
- 2022A Generalist Agentcited by 808
- 2019ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Taskscited by 783
- 2023VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Modelscited by 750
- 2021CLIPort: What and Where Pathways for Robotic Manipulationcited by 622
- 2022ProgPrompt: Generating Situated Robot Task Plans using Large Language Modelscited by 611
- 2023Prompt Injection attack against LLM-integrated Applicationscited by 605
- 2023API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMscited by 591
- 2018VirtualHome: Simulating Household Activities via Programscited by 415
- 2022VIMA: General Robot Manipulation with Multimodal Promptscited by 407
- 2022Visual Language Maps for Robot Navigationcited by 365
- 2023SayPlan: Grounding Large Language Models using 3D Scene Graphs for Scalable Robot Task Planningcited by 294
- 2016Control Barrier Function Based Quadratic Programs for Safety Critical Systemscited by 287
- 2023Benchmarking and Defending Against Indirect Prompt Injection Attacks on Large Language Modelscited by 264
- 2021TEACh: Task-driven Embodied Agents that Chatcited by 197
50 most cited of 54 All 54 (JSON)
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2024Compromising Embodied Agents with Contextual Backdoor Attacks
- 2026Jailbreaking Embodied LLMs via Action-level Manipulation
- 2024Can We Trust Embodied Agents? Exploring Backdoor Attacks against Embodied LLM-based Decision-Making Systems
- 2026Security of Foundation-Model-Powered Embodied Agents: Attack Surfaces, Attacks, Defenses, and Evaluation
- 2026RPMS: Enhancing LLM-Based Embodied Planning through Rule-Augmented Memory Synergy
- 2026Security and Privacy in Large-Model-Driven Embodied Agents: Attacks, Defenses, and Future Directions
- 2025Towards Robust and Secure Embodied AI: A Survey on Vulnerabilities and Attacks
- 2026Security of World-Model-Based Embodied AI: A Lifecycle of Threats, Defenses, and Evaluation