Adversarial Attacks in Multi-Agent LLM Pipelines: Unveiling Structural Vulnerabilities in Agentic AI Architectures
On the map: LLM agents and tool use → Agent threats, guardrails and governance
Abstract (arXiv)
Multi-agent LLM pipelines orchestrate multiple specialized language model agents into structured workflows where intermediate outputs are passed across agents to solve complex tasks. This design introduces a security gap absent in single-agent settings: once an agent accepts adversarial content, it is propagated as trusted input throughout the pipeline. We argue that this vulnerability stems from the absence of boundary verification, a security primitive that enforces explicit validation of data as it crosses inter-agent boundaries, including content, identity, execution intent, and state integrity. Without such verification, modern pipelines embed implicit trust assumptions that are not adversarially robust, giving rise to structurally distinct attack surfaces (e.g., content injection, agent impersonation, plan deviation, and memory poisoning). Leveraging annotated production traces from the GAIA and SWE-Bench benchmark, we show that these vulnerabilities arise in benign deployments and largely evade existing evaluation frameworks. We further operationalize these failure modes within a controlled multi-agent setting and evaluate them across GPT-5-mini, Claude Sonnet 4.5, and Kimi K2.5 under identical pipeline configurations. The results reveal that attack success aligns with pipeline structure rather than model capability, indicating that adversarial vulnerability is fundamentally an architectural property and motivating a shift toward pipeline-level defenses.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 22 of its 29 references that are in the map. First posted on arXiv 1 Aug 2026.
Builds on
22 of 29 references.
Closest ancestors · 11
Same direction, newest first
- 2025TRiSM for Agentic AI: A Review of Trust, Risk, and Security Management in LLM-based Agentic Multi-Agent Systems4 Jun
- 2025textitAgents Under Siege: Breaking Pragmatic Multi-Agent LLM Systems with Optimized Prompt Attacks31 Mar
- 2024Prompt Infection: LLM-to-LLM Prompt Injection within Multi-Agent Systems9 Oct
- 2024Agent Security Bench (ASB): Formalizing and Benchmarking Attacks and Defenses in LLM-based Agents3 Oct
- 2024Audit-LLM: Multi-Agent Collaboration for Log-based Insider Threat Detection12 Aug
- 2024The Emerged Security and Privacy of LLM Agent: A Survey with Case Studies28 Jul
- 2024Flooding Spread of Manipulated Knowledge in LLM-Based Multi-Agent Communities10 Jul
- 2024AutoDefense: Multi-Agent LLM Defense against Jailbreak Attacks2 Mar
- 2024Agent Smith: A Single Image Can Jailbreak One Million Multimodal LLM Agents Exponentially Fast13 Feb
- 2024Secret Collusion among AI Agents: Multi-Agent Deception via Steganography12 Feb
- 2024PsySafe: A Comprehensive Framework for Psychological-based Attack, Defense, and Evaluation of Multi-agent System Safety22 Jan
Other references · 11
- 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
- 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
- 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,906
- 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,339
- 2023GAIA: a benchmark for General AI Assistantscited by 1,084
- 2023Formalizing and Benchmarking Prompt Injection Attacks and Defensescited by 328
- 2024PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language Modelscited by 251
- 2024AI Agents Under Threat: A Survey of Key Security Challenges and Future Pathwayscited by 122
- 2025The Attacker Moves Second: Stronger Adaptive Attacks Bypass Defenses Against Llm Jailbreaks and Prompt Injectionscited by 108
- 2025TRAIL: Trace Reasoning and Agentic Issue Localizationcited by 98
- 2025Open Challenges in Multi-Agent Security: Towards Secure Systems of Interacting AI Agentscited by 66
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2025Breaking Agent Backbones: Evaluating the Security of Backbone LLMs in AI Agents
- 2025Exposing Weak Links in Multi-Agent Systems under Adversarial Prompting
- 2026Trust propagation and structural containment in Multi-agent LLM pipelines
- 2025TAMAS: Benchmarking Adversarial Risks in Multi-Agent LLM Systems
- 2026Agent-Fence: Mapping Security Vulnerabilities Across Deep Research Agents
- 2026DUMA-Bench: A Dual-Control Multi-Agent Benchmark for Evaluating LLM Agent Security
- 2026ChannelGuard: Safe Models Do Not Compose into Safe Multi-Agent Systems
- 2026A Systematic Survey of Security Threats and Defenses in LLM-Based AI Agents: A Layered Attack Surface Framework