Don't Trust Your Upstream: Exploiting LLM Multi-Agent System via Topology-Guided Adversarial Propagation
On the map: LLM agents and tool use → Agent threats, guardrails and governance
Abstract (arXiv)
The digital world is witnessing the rapid rise of LLM-based multi-agent systems (MASs) and their powerful applications. However, their security remains insufficiently understood, as existing evaluations are largely limited to narrow attack settings and may substantially underestimate the real risks of MAS deployments. Inspired by the MAS inter-agent dependencies, where upstream outputs are reinterpreted and executed by downstream agents, we propose a topology-aware attack scheme that propagates adversarial contamination from exposed edge agents to high-privilege agents to induce malicious behaviors. By combining topology reconnaissance, contamination propagation modeling, and hierarchical payload encapsulation, our approach overcomes the key challenges of black-box attacks and makes such multi-hop compromise practical. Experiments show that our approach achieves success rates of 40%--78% on three widely-used MAS frameworks under five topologies, and 85% on two real-world MAS applications across 20 representative scenarios. The results reveal fundamental vulnerabilities in MASs that have been overlooked by prior studies. Based on these findings, we propose a topology-trust mitigation that blocks 94.8% of such composite attacks.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 40 of its 57 references that are in the map. First posted on arXiv 3 Dec 2025.
Builds on
40 of 57 references.
Closest ancestors · 14
Same direction, newest first
- 2025Web Fraud Attacks Against LLM-Driven Multi-Agent Systems1 Sep
- 2025IP Leakage Attacks Targeting LLM-Based Multi-Agent Systems18 May
- 2025AgentSafe: Safeguarding Large Language Model-based Multi-agent Systems via Hierarchical Data Management6 Mar
- 2025Red-Teaming LLM Multi-Agent Systems via Communication Attacks20 Feb
- 2025Automating Prompt Leakage Attacks on Large Language Models Using Agentic Approach18 Feb
- 2025Unveiling Privacy Risks in LLM Agent Memory17 Feb
- 2025AGrail: A Lifelong Agent Guardrail with Effective and Adaptive Safety Detection17 Feb
- 2025G-Safeguard: A Topology-Guided Security Lens and Treatment on LLM-based Multi-agent Systems16 Feb
- 2024Attacking Vision-Language Computer Agents via Pop-ups4 Nov
- 2024EIA: Environmental Injection Attack on Generalist Web Agents for Privacy Leakage17 Sep
- 2024AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Bases17 Jul
- 2024Flooding Spread of Manipulated Knowledge in LLM-Based Multi-Agent Communities10 Jul
2 more
Other references · 26
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
- 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
- 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,168
- 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,906
- 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,134
- 2023LIMA: Less Is More for Alignmentcited by 1,080
- 2023AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviorscited by 580
- 2023"Do Anything Now": Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Modelscited by 457
- 2024Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attackcited by 364
- 2024Magentic-One: A Generalist Multi-Agent System for Solving Complex Taskscited by 221
- 2024TradingAgents: Multi-Agents LLM Financial Trading Frameworkcited by 182
14 more
- 2024MDAgents: An Adaptive Collaboration of LLMs for Medical Decision-Makingcited by 162
- 2024Cut the Crap: An Economical Communication Pipeline for LLM-based Multi-Agent Systemscited by 129
- 2025OWL: Optimized Workforce Learning for General Multi-Agent Assistance in Real-World Task Automationcited by 129
- 2023ChatEDA: A Large Language Model Powered Autonomous Agent for EDAcited by 118
- 2024AgentOccam: A Simple Yet Strong Baseline for LLM-Based Web Agentscited by 110
- 2025Multi-Agent Design: Optimizing Agents with Better Prompts and Topologiescited by 106
- 2024On the Resilience of LLM-Based Multi-Agent Collaboration with Faulty Agentscited by 92
- 2024ProtAgents: Protein discovery via large language model multi-agent collaborations combining physics and machine learningcited by 63
- 2024Caution for the Environment: Multimodal LLM Agents are Susceptible to Environmental Distractionscited by 55
- 2024MultiAgent Collaboration Attack: Investigating Adversarial Attacks in Large Language Model Collaborations via Debatecited by 54
- 2025Understanding the Information Propagation Effects of Communication Topologies in LLM-based Multi-Agent Systemscited by 39
- 2024Denial-of-Service Poisoning Attacks against Large Language Modelscited by 34
- 2025Mirage in the Eyes: Hallucination Attack on Multi-modal Large Language Models with Only Attention Sinkcited by 15
- 2025Topological Structure Learning Should Be A Research Priority for LLM-Based Multi-Agent Systemscited by 5
Built on by
Citing papers per year: 2026: 3.
3 papers.
Similar, not cited
Close in wording, no citation either way
- 2026Adversarial Attacks in Multi-Agent LLM Pipelines: Unveiling Structural Vulnerabilities in Agentic AI Architectures
- 2026CIA: Inferring the Communication Topology from LLM-based Multi-Agent Systems
- 2025Securing Multi-Agent Systems Against Corruptions via Node Contribution Backpropagation
- 2026WebWeaver: Breaking Topology Confidentiality in LLM Multi-Agent Systems with Stealthy Context-Based Inference
- 2026PropGuard: Safeguarding LLM-MAS via Propagation-Aware Exploration and Remediation
- 2026Conjunctive Prompt Attacks in Multi-Agent LLM Systems
- 2025Multi-Agent Systems Execute Arbitrary Malicious Code
- 2025The Dark Side of LLMs: Agent-based Attack Vectors for System-level Compromise