# Trust No Tool: Evaluating and Defending LLM Agents under Untrusted Tool Feedback arXiv: https://arxiv.org/abs/2605.17453 ยท HTML: https://mlsanity.com/paper/2605.17453/ ## Key facts - Cited by 2 papers in the map (data through 14 Oct 2026). - Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. - Builds on 40 of its 41 references that are in the map. - First posted on arXiv 17 May 2026. ## Placed in - LLM agents and tool use / [Agent threats, guardrails and governance](https://mlsanity.com/agents/agent-threats-guardrails-and-governance/) ## Builds on (most cited first, 30 of 40) - [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024) - [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025) - [Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection](https://mlsanity.com/paper/2302.12173/) (2023) - [Reasoning with Language Model is Planning with World Model](https://mlsanity.com/paper/2305.14992/) (2023) - [Identifying the Risks of LM Agents with an LM-Emulated Sandbox](https://mlsanity.com/paper/2309.15817/) (2023) - [InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents](https://mlsanity.com/paper/2403.02691/) (2024) - [AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents](https://mlsanity.com/paper/2410.09024/) (2024) - [Agent Security Bench (ASB): Formalizing and Benchmarking Attacks and Defenses in LLM-based Agents](https://mlsanity.com/paper/2410.02644/) (2024) - [Defeating Prompt Injections by Design](https://mlsanity.com/paper/2503.18813/) (2025) - [ToolSandbox: A Stateful, Conversational, Interactive Evaluation Benchmark for LLM Tool Use Capabilities](https://mlsanity.com/paper/2408.04682/) (2024) - [Benchmarking and Defending Against Indirect Prompt Injection Attacks on Large Language Models](https://mlsanity.com/paper/2312.14197/) (2023) - [Leveraging Pre-trained Large Language Models to Construct and Utilize World Models for Model-based Task Planning](https://mlsanity.com/paper/2305.14909/) (2023) - [StableToolBench: Towards Stable Large-Scale Benchmarking on Tool Learning of Large Language Models](https://mlsanity.com/paper/2403.07714/) (2024) - [The Task Shield: Enforcing Task Alignment to Defend Against Indirect Prompt Injection in LLM Agents](https://mlsanity.com/paper/2412.16682/) (2024) - [CWM: An Open-Weights LLM for Research on Code Generation with World Models](https://mlsanity.com/paper/2510.02387/) (2025) - [The Tool Decathlon: Benchmarking Language Agents for Diverse, Realistic, and Long-Horizon Task Execution](https://mlsanity.com/paper/2510.25726/) (2025) - [OpenAgentSafety: A Comprehensive Framework for Evaluating Real-World AI Agent Safety](https://mlsanity.com/paper/2507.06134/) (2025) - [ToolSafe: Enhancing Tool Invocation Safety of LLM-based agents via Proactive Step-level Guardrail and Feedback](https://mlsanity.com/paper/2601.10156/) (2026) - [MCPTox: A Benchmark for Tool Poisoning Attack on Real-World MCP Servers](https://mlsanity.com/paper/2508.14925/) (2025) - [ImpossibleBench: Measuring LLMs' Propensity of Exploiting Test Cases](https://mlsanity.com/paper/2510.20270/) (2025) - [RedTeamCUA: Realistic Adversarial Testing of Computer-Use Agents in Hybrid Web-OS Environments](https://mlsanity.com/paper/2505.21936/) (2025) - [Generating Code World Models with Large Language Models Guided by Monte Carlo Tree Search](https://mlsanity.com/paper/2405.15383/) (2024) - [MCPMark: A Benchmark for Stress-Testing Realistic and Comprehensive MCP Use](https://mlsanity.com/paper/2509.24002/) (2025) - [MCP-SafetyBench: A Benchmark for Safety Evaluation of Large Language Models with Real-World MCP Servers](https://mlsanity.com/paper/2512.15163/) (2025) - [Unsafer in Many Turns: Benchmarking and Defending Multi-Turn Safety Risks in Tool-Using Agents](https://mlsanity.com/paper/2602.13379/) (2026) - [AgentSentry: Mitigating Indirect Prompt Injection in LLM Agents via Temporal Causal Diagnostics and Context Purification](https://mlsanity.com/paper/2602.22724/) (2026) - [BrowseSafe: Understanding and Preventing Prompt Injection Within AI Browser Agents](https://mlsanity.com/paper/2511.20597/) (2025) - [Breaking the Protocol: Security Analysis of the Model Context Protocol Specification and Prompt Injection Vulnerabilities in Tool-Integrated LLM Agents](https://mlsanity.com/paper/2601.17549/) (2026) - [SoK: The Attack Surface of Agentic AI - Tools and Autonomy](https://mlsanity.com/paper/2603.22928/) (2026) - [MCP-ITP: An Automated Framework for Implicit Tool Poisoning in MCP](https://mlsanity.com/paper/2601.07395/) (2026) ## Built on by (most cited first, 2 of 2) - [MemToC: Benchmarking Memory-Tool Conflict Resolution in Large Language Models](https://mlsanity.com/paper/2608.26295/) (2026) - [Evidence Blindness in Direct Corpus Interaction: Persistent Navigation with AtlasNav](https://mlsanity.com/paper/2608.24764/) (2026) ## Abstract (arXiv) Tool-using LLM agents increasingly rely on external tools to make consequential decisions, yet most existing agent-security benchmarks and defenses implicitly assume that tool feedback is trustworthy once a tool has been selected. We study a different failure mode, cognitive poisoning, in which a malicious tool behaves plausibly during exploration, accumulates trust through benign-looking feedback, and becomes harmful only when hidden state conditions align with the final executable action. To study this setting, we construct TRUST-Bench, a task-conditioned benchmark of 1,970 hidden-trigger tool-compromise episodes with matched safe controls, introduce an asymmetric penalty metric, GuardedJoint, to better reflect real deployment risk, and present VISTA-Guard, a backbone-agnostic framework for final-action risk scoring. The core idea is to abstract multi-step tool interaction into structured environment variables that encode trust-formation dynamics and then score the risk of the final executable action from this trajectory-conditioned representation. Experiments show that prompt-centric heuristics, scalarized features, and zero-shot judges fail in this regime, whereas trajectory-aware final-action scoring yields strong in-domain discrimination and remains effective under balanced out-of-distribution transfer. Under GuardedJoint, VISTA-Guard reaches 84.2 in-domain and 56.9 on balanced out-of-distribution evaluation, while methods that optimize only one side of the safety--utility tradeoff collapse to zero. These findings support a broader view of agent security in black-box tool ecosystems: the decisive defense target is not local prompt text or tool descriptors alone, but the way trust is formed across the interaction trajectory and committed through the final action.