The Trust Paradox in LLM-Based Multi-Agent Systems: When Collaboration Becomes a Security Vulnerability
On the map: LLM agents and tool use → Agent threats, guardrails and governance
Abstract (arXiv)
Multi-agent systems powered by large language models are advancing rapidly, yet the tension between mutual trust and security remains underexplored. We introduce and empirically validate the Trust-Vulnerability Paradox (TVP): increasing inter-agent trust to enhance coordination simultaneously expands risks of over-exposure and over-authorization. To investigate this paradox, we construct a scenario-game dataset spanning 3 macro scenes and 19 sub-scenes, and run extensive closed-loop interactions with trust explicitly parameterized. Using Minimum Necessary Information (MNI) as the safety baseline, we propose two unified metrics: Over-Exposure Rate (OER) to detect boundary violations, and Authorization Drift (AD) to capture sensitivity to trust levels. Results across multiple model backends and orchestration frameworks reveal consistent trends: higher trust improves task success but also heightens exposure risks, with heterogeneous trust-to-risk mappings across systems. We further examine defenses such as Sensitive Information Repartitioning and Guardian-Agent enablement, both of which reduce OER and attenuate AD. Overall, this study formalizes TVP, establishes reproducible baselines with unified metrics, and demonstrates that trust must be modeled and scheduled as a first-class security variable in multi-agent system design.
Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 25 of its 47 references that are in the map. First posted on arXiv 21 Oct 2025.
Builds on
25 of 47 references.
Closest ancestors · 4
Same direction, newest first
- 2025Towards Transparent and Incentive-Compatible Collaboration in Decentralized LLM Multi-Agent Systems: A Blockchain-Driven Approach20 Sep
- 2025Security of Internet of Agents: Attacks and Countermeasures12 May
- 2025A Survey on Trustworthy LLM Agents: Threats and Countermeasures12 Mar
- 2024AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Bases17 Jul
Other references · 21
- 2023GPT-4 Technical Reportcited by 18,132
- 2023Qwen Technical Reportcited by 3,273
- 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
- 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 1,348
- 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,339
- 2023Large Language Models Empowered Agent-based Modeling and Simulation: A Survey and Perspectivescited by 253
- 2023Benchmarking and Defending Against Indirect Prompt Injection Attacks on Large Language Modelscited by 237
- 2024Security and Privacy Challenges of Large Language Models: A Surveycited by 179
- 2024LLM-Based Multi-Agent Systems for Software Engineering: Literature Review, Vision and the Road Aheadcited by 155
- 2025Evaluation and Benchmarking of LLM Agents: A Surveycited by 145
- 2024AI Agents Under Threat: A Survey of Key Security Challenges and Future Pathwayscited by 122
- 2024AgentScope: A Flexible yet Robust Multi-Agent Platformcited by 120
9 more
- 2023Scalable Multi-Robot Collaboration with Large Language Models: Centralized or Decentralized Systems?cited by 104
- 2024AgentGen: Enhancing Planning Abilities for Large Language Model based Agent via Environment and Task Generationcited by 80
- 2024Agents in Software Engineering: Survey, Landscape, and Visioncited by 60
- 2025Plan-Then-Execute: An Empirical Study of User Trust and Team Performance When Using LLM Agents As A Daily Assistantcited by 54
- 2025LLM-Based Human-Agent Collaboration and Interaction Systems: A Surveycited by 28
- 2025Multi-Agent Security Tax: Trading Off Security and Collaboration Capabilities in Multi-Agent Systemscited by 27
- 2024Large Model Based Agents: State-of-the-Art, Cooperation Paradigms, Security and Privacy, and Future Trendscited by 26
- 2024Unique Security and Privacy Threats of Large Language Models: A Comprehensive Surveycited by 20
- 2025Shapley-Coop: Credit Assignment for Emergent Cooperation in Self-Interested LLM Agentscited by 11
Built on by
Citing papers per year: 2025: 1; 2026: 10.
11 papers.
- 2025Security Analysis of Agentic AI Communication Protocols: A Comparative Evaluationcited by 6
- 2026OpenKedge: Governing Agentic Mutation with Execution-Bound Safety and Evidence Chainscited by 4
- 2026Do Coding Agents Understand Least-Privilege Authorization?cited by 4
- 2026Conjunctive Prompt Attacks in Multi-Agent LLM Systemscited by 2
- 2026SoK: When Safe Agents Fail Together: The Security of Multi Agent LLM Systemscited by 1
- 2026Conversations Risk Detection LLMs in Financial Agents via Multi-Stage Generative Rolloutcited by 0
- 2026When Should Agent Trust Be Conditional? Characterizing and Attacking Skill-Conditional Reputation in Agent Swarmscited by 0
- 2026From Parasocial Scripts to Dyadic Persistence in Autonomous AI-Agent Communitiescited by 0
- 2026Data Leakage Prevention in Agentic Applications via Preemptive Hardeningcited by 0
- 2026From Monoliths to Swarms: A Study of Attack Surface Evolution in the Transition to Multi-Agent Web Systemscited by 0
- 2026A Policy Algebra for Trust-Preserving Agentic AI Executioncited by 0
- 2026SoK: When Safe Agents Fail Together: The Security of Multi Agent LLM Systems1 Sep 2026
- 2026A Policy Algebra for Trust-Preserving Agentic AI Execution17 Aug 2026
- 2026From Monoliths to Swarms: A Study of Attack Surface Evolution in the Transition to Multi-Agent Web Systems31 Jul 2026
- 2026Data Leakage Prevention in Agentic Applications via Preemptive Hardening21 Jul 2026
- 2026From Parasocial Scripts to Dyadic Persistence in Autonomous AI-Agent Communities15 Jun 2026
- 2026When Should Agent Trust Be Conditional? Characterizing and Attacking Skill-Conditional Reputation in Agent Swarms12 Jun 2026
- 2026Do Coding Agents Understand Least-Privilege Authorization?14 May 2026
- 2026Conjunctive Prompt Attacks in Multi-Agent LLM Systems17 Apr 2026
- 2026Conversations Risk Detection LLMs in Financial Agents via Multi-Stage Generative Rollout10 Apr 2026
- 2026OpenKedge: Governing Agentic Mutation with Execution-Bound Safety and Evidence Chains7 Apr 2026
- 2025Security Analysis of Agentic AI Communication Protocols: A Comparative Evaluation5 Nov 2025
Similar, not cited
Close in wording, no citation either way
- 2025Comprehensive Vulnerability Analysis is Necessary for Trustworthy LLM-MAS
- 2025To trust or not to trust: Attention-based Trust Management for LLM Multi-Agent Systems
- 2026Can Agents Trust Their Skills? Uncovering Unsafe Chains of Trust in Skill-Based LLM Agents
- 2025TRiSM for Agentic AI: A Review of Trust, Risk, and Security Management in LLM-based Agentic Multi-Agent Systems
- 2026Trust No Tool: Evaluating and Defending LLM Agents under Untrusted Tool Feedback
- 2026"Are You Sure?": An Empirical Study of Human Perception Vulnerability in LLM-Driven Agentic Systems
- 2025Seven Security Challenges in Cross-domain Multi-agent LLM Systems
- 2025SoK: Trust-Authorization Mismatch in LLM Agent Interactions