mlsanityData to 14 Oct 2026

The Surface You Test Is Not the Surface That Breaks

arXiv 2605.30454 ↗First posted 28 May 2026cs.CR cs.AIcited by 0

On the map: LLM agents and tool use → Agent threats, guardrails and governance

Abstract (arXiv)

Prompt-injection benchmarks for LLM agents typically test attacks through a single injection surface and report the resulting attack success rate as a property of the model. We ask whether those robustness conclusions remain stable when the same adversarial content enters through a different part of the agent interface. Using AgentDojo, we evaluate 13 LLMs across four task suites and place a byte-identical payload either in a tool output or in the tool description. This small change produces large differences in comparative robustness: 44.9% of all model pairs change their relative ordering across the two surfaces, with substantial ranking instability in every suite. The effect is especially pronounced for a small number of models, showing that a benchmark can substantially underestimate vulnerability when it tests only one surface. We further find that this behavior has predictive structure. Using three suites to identify the riskier surface for each model predicts the more vulnerable surface on an unseen suite with 76.9% accuracy. Defense results show the same dependence, as mitigations effective against tool-output attacks can leave substantial exposure through tool descriptions. Our results show that prompt-injection robustness is not surface-invariant and that agent evaluations should test the surfaces on which their security conclusions depend.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 26 of its 30 references that are in the map. First posted on arXiv 28 May 2026.

Builds on

26 of 30 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025Prompt Injection Attack to Tool Selection in LLM Agents28 Apr
  2. 2025Adaptive Attacks Break Defenses Against Indirect Prompt Injection Attacks on LLM Agents27 Feb
  3. 2024From Allies to Adversaries: Manipulating LLM Tool-Calling through Adversarial Injection13 Dec
  4. 2024InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents5 Mar

Other references · 22

  1. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  2. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  3. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,947
  4. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,651
  5. 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,451
  6. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,443
  7. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,420
  8. 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,332
  9. 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,268
  10. 2022Ignore Previous Prompt: Attack Techniques For Language Modelscited by 832
  11. 2020On Adaptive Attacks to Adversarial Example Defensescited by 555
  12. 2024The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructionscited by 493
10 more
  1. 2024Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attackscited by 486
  2. 2023SmoothLLM: Defending Large Language Models Against Jailbreaking Attackscited by 469
  3. 2019On Evaluating Adversarial Robustnesscited by 447
  4. 2024Defending Against Indirect Prompt Injection Attacks With Spotlightingcited by 267
  5. 2023Benchmarking and Defending Against Indirect Prompt Injection Attacks on Large Language Modelscited by 264
  6. 2023Defending Against Alignment-Breaking Attacks via Robustly Aligned LLMcited by 192
  7. 2024Stealing Part of a Production Language Modelcited by 159
  8. 2025The Attacker Moves Second: Stronger Adaptive Attacks Bypass Defenses Against Llm Jailbreaks and Prompt Injectionscited by 129
  9. 2023Abusing Images and Sounds for Indirect Instruction Injection in Multi-Modal LLMscited by 124
  10. 2025System Prompt Poisoning: Persistent Attacks on Large Language Models Beyond User Injectioncited by 14

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025More Vulnerable than You Think: On the Stability of Tool-Integrated LLM Agents
  2. 2026Skill-Inject: Measuring Agent Vulnerability to Skill File Attacks
  3. 2026What Makes a Good LLM Agent for Real-world Penetration Testing?
  4. 2026Mapping the Exploitation Surface: A 10,000-Trial Taxonomy of What Makes LLM Agents Exploit Vulnerabilities
  5. 2026Kill-Chain Canaries: Stage-Level Tracking of Prompt Injection Across Attack Surfaces and Model Safety Tiers
  6. 2026Passing the Test You Trained On: Re-evaluating Prompt-Injection Detectors for LLM Agents
  7. 2026The Framing Gap: Indirect Prompt-Injection Exfiltration Defeats Surface-Level Defenses in Tool-Using Agents
  8. 2026Adaptive Evaluation of Out-of-Band Defenses Against Prompt Injection in LLM Agents