Think Twice Before You Act: Protecting LLM Agents Against Tool Description Poisoning via Isolated Planning
On the map: LLM agents and tool use → Agent threats, guardrails and governance
also in 1 more
Abstract (arXiv)
The integration of external tools has substantially expanded the capabilities of large language model (LLM) agents, but it also introduces new attack surfaces beyond prompt injection. In particular, cross-tool description poisoning can manipulate planner-visible tool metadata to steer an agent's trajectory, even if the poisoned tool itself is never chosen. To understand the effectiveness of existing defenses against this emerging threat, we first evaluate several prompt-injection defenses and find that they transfer poorly to cross-tool description poisoning. A key observation is that poisoned descriptions persist in the planning context across steps, enabling continuous influence over subsequent tool choices. Building on this insight, we propose Tool-Guard, a novel system-level defense based on a new concept called isolated planning, in which tool invocations that are detected as misaligned or suspicious cause the corresponding tool to be placed in a quarantined list (the influenced list), breaking further influence from poisoned descriptions. With this influence isolated, the tool can continue to be used to support the task, enabling a robust defense that preserves legitimate tool utility. Experiments on the AgentDojo and ASB benchmarks show that Tool-Guard substantially reduces attack success while maintaining high task utility. Our code is available at https://github.com/shishishi123/Tool-Guard.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 19 of its 46 references that are in the map. First posted on arXiv 18 Jun 2026.
Builds on
19 of 46 references.
Closest ancestors · 8
Same direction, newest first
- 2025ToolTweak: An Attack on Tool Selection in LLM-based Agents2 Oct
- 2025IPIGuard: A Novel Tool Dependency Graph-Based Defense Against Indirect Prompt Injection in LLM Agents21 Aug
- 2025MCPTox: A Benchmark for Tool Poisoning Attack on Real-World MCP Servers19 Aug
- 2025Attractive Metadata Attack: Inducing LLM Agents to Invoke Malicious Tools4 Aug
- 2025DRIFT: Dynamic Rule-Based Defense with Injection Isolation for Securing LLM Agents13 Jun
- 2025Defeating Prompt Injections by Design24 Mar
- 2025MELON: Provable Defense Against Indirect Prompt Injection Attacks in AI Agents7 Feb
- 2024Agent Security Bench (ASB): Formalizing and Benchmarking Attacks and Defenses in LLM-based Agents3 Oct
Other references · 11
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
- 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,339
- 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,326
- 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,134
- 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,124
- 2022Ignore Previous Prompt: Attack Techniques For Language Modelscited by 760
- 2023Tree of Attacks: Jailbreaking Black-Box LLMs Automaticallycited by 619
- 2023Prompt Injection attack against LLM-integrated Applicationscited by 568
- 2023A Real-World WebAgent with Planning, Long Context Understanding, and Program Synthesiscited by 350
- 2024StruQ: Defending Against Prompt Injection with Structured Queriescited by 325
- 2025MCIP: Protecting MCP Safety via Model Contextual Integrity Protocolcited by 33
Built on by
Citing papers per year: 2026: 3.
3 papers.
- 2026Convergent Detour Hijacking: Task-Preserving Resource Amplification in Skill-Based LLM Agentscited by 2
- 2026ECLIPSE: Self-Evolving Stealthy Prompt Injection Attack against Long-Horizon Agentic Systemscited by 0
- 2026Skynet: Workflow-Level Anomaly Detection for Agentic AI via Semantic and Structural Modelingcited by 0
- 2026Skynet: Workflow-Level Anomaly Detection for Agentic AI via Semantic and Structural Modeling6 Sep 2026
- 2026ECLIPSE: Self-Evolving Stealthy Prompt Injection Attack against Long-Horizon Agentic Systems31 Aug 2026
- 2026Convergent Detour Hijacking: Task-Preserving Resource Amplification in Skill-Based LLM Agents12 Aug 2026
Similar, not cited
Close in wording, no citation either way
- 2026When the Manual Lies: A Realistic Benchmark to Evaluate MCP Poisoning Attacks for LLM Agents
- 2026Universal Defenses for Tool-Integrated LLM Agents Against Adversarial Attacks
- 2026PlanGuard: Defending Agents against Indirect Prompt Injection via Planning-based Consistency Verification
- 2026SafeClawBench: Separating Semantic, Audit-Evidence, and Sandbox Harm in Tool-Using LLM Agents
- 2026ToolSafe: Enhancing Tool Invocation Safety of LLM-based agents via Proactive Step-level Guardrail and Feedback
- 2026ClawGuard: A Runtime Security Framework for Tool-Augmented LLM Agents Against Indirect Prompt Injection
- 2025MindGuard: Intrinsic Decision Inspection for Securing LLM Agents Against Metadata Poisoning
- 2025Les Dissonances: Cross-Tool Harvesting and Polluting in Pool-of-Tools Empowered LLM Agents