mlsanityData to 14 Oct 2026

Measuring and Exploiting Implicit Trust in LLM Tool-Calling Pipelines

arXiv 2609.18217 ↗First posted 16 Sep 2026cs.CRcited by 0

On the map: LLM agents and tool use → Agent threats, guardrails and governance

Abstract (arXiv)

The Model Context Protocol (MCP) enables LLMs to invoke external tools, but every tool interaction exposes the model to attacker-controlled text through multiple input channels (tool descriptions, tool results, sampling messages) that share a single context window without privilege separation. In this paper, we present a framework to measure the trust profile of an arbitrary LLM based on a variety of payload framings sent through different channels. Following this assessment, we devise cross-channel fragmentation attacks that distribute seemingly benign payloads across two or three channels; no individual channel carries a complete injection, yet the LLM compiles the fragments into credential exfiltration. We evaluated our attacks across 12 frontier models, three production clients, and six payloads, totalling over 15,000 trials. Our evaluation reveals that cross-channel attacks are an unexplored attack surface: models that fully resist single-channel injection (0% compliance) exfiltrate sensitive data at up to 100% under two-channel fragmentation (e.g., GPT-4o, Llama 70B, Composer 2, Haiku 4.5). We further demonstrate value-aligned exploitation, where a tool's stated purpose requires the data the attacker targets, and a sampling system prompt override that injects persistent instructions via VS Code's MCP implementation. Finally, we evaluated our attacks against seven third-party MCP security tools and three prompt-based defenses. All tools failed to detect fragmented payloads, and prompt defenses proved model-specific rather than universal.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 26 of its 51 references that are in the map. First posted on arXiv 16 Sep 2026.

Builds on

26 of 51 references.

Closest ancestors · 15

Same direction, newest first

  1. 2026AgentLAB: Benchmarking LLM Agents against Long-Horizon Attacks18 Feb
  2. 2026Security Threat Modeling for Emerging AI-Agent Protocols: A Comparative Analysis of MCP, A2A, Agora, and ANP11 Feb
  3. 2026Breaking the Protocol: Security Analysis of the Model Context Protocol Specification and Prompt Injection Vulnerabilities in Tool-Integrated LLM Agents24 Jan
  4. 2026Prompt Injection Attacks on Agentic Coding Assistants: A Systematic Analysis of Vulnerabilities in Skills, Tools, and Protocol Ecosystems24 Jan
  5. 2026Taming Various Privilege Escalation in LLM-Based Agent Systems: A Mandatory Access Control Framework17 Jan
  6. 2025Attention is All You Need to Defend Against Indirect Prompt Injection Attacks in LLMs9 Dec
  7. 2025A First Look at the Security Issues in the Model Context Protocol Ecosystem18 Oct
  8. 2025MCP Security Bench (MSB): Benchmarking Attacks Against Model Context Protocol in LLM Agents14 Oct
  9. 2025MCPSecBench: A Systematic Security Benchmark and Playground for Testing Model Context Protocols17 Aug
  10. 2025Prompt Injection Attack to Tool Selection in LLM Agents28 Apr
  11. 2025Les Dissonances: Cross-Tool Harvesting and Polluting in Pool-of-Tools Empowered LLM Agents4 Apr
  12. 2025MCP Safety Audit: LLMs with the Model Context Protocol Allow Major Security Exploits2 Apr
3 more
  1. 2025Defeating Prompt Injections by Design24 Mar
  2. 2024Agent Security Bench (ASB): Formalizing and Benchmarking Attacks and Defenses in LLM-based Agents3 Oct
  3. 2024InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents5 Mar

Other references · 11

  1. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,443
  2. 2024The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructionscited by 493
  3. 2024StruQ: Defending Against Prompt Injection with Structured Queriescited by 375
  4. 2023Formalizing and Benchmarking Prompt Injection Attacks and Defensescited by 364
  5. 2025Model Context Protocol (MCP): Landscape, Security Threats, and Future Research Directionscited by 265
  6. 2024SecAlign: Defending Against Prompt Injection with Preference Optimizationcited by 210
  7. 2025DataSentinel: A Game-Theoretic Detection of Prompt Injection Attackscited by 143
  8. 2025The Attacker Moves Second: Stronger Adaptive Attacks Bypass Defenses Against Llm Jailbreaks and Prompt Injectionscited by 129
  9. 2024PLeak: Prompt Leaking Attacks against Large Language Model Applicationscited by 127
  10. 2026The Promptware Kill Chain: How Prompt Injections Gradually Evolved Into a Multistep Malware Delivery Mechanismcited by 9
  11. 2025Trivial Trojans: How Minimal MCP Servers Enable Cross-Tool Exfiltration of Sensitive Datacited by 6

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Same Payload, Different Channel: Measuring Trust Asymmetry in Tool-Using Language Models
  2. 2026Attesting LLM Pipelines: Enforcing Verifiable Training and Release Claims
  3. 2026Beyond Max Tokens: Stealthy Resource Amplification via Tool Calling Chains in LLM Agents
  4. 2024From Allies to Adversaries: Manipulating LLM Tool-Calling through Adversarial Injection
  5. 2025Semantic Attacks on Tool-Augmented LLMs: Securing the Model Context Protocol Against Descriptor-Level Manipulation
  6. 2026PACE: Provenance-Aware Capability Enforcement for Tool-Using LLM Agents
  7. 2025Parasites in the Toolchain: A Large-Scale Analysis of Attacks on the MCP Ecosystem
  8. 2026You Told Me to Do It: Measuring Instructional Text-induced Private Data Leakage in LLM Agents