To trust or not to trust: Attention-based Trust Management for LLM Multi-Agent Systems
On the map: LLM agents and tool use → Agent threats, guardrails and governance
Abstract (arXiv)
Large Language Model-based Multi-Agent Systems (LLM-MAS) have demonstrated strong capabilities in solving complex tasks but remain vulnerable when agents receive unreliable messages. This vulnerability stems from a fundamental gap: LLM agents treat all incoming messages equally without evaluating their trustworthiness. While some existing studies approach trustworthiness, they focus on a single type of harmfulness rather than analyze it in a holistic approach from multiple trustworthiness perspectives. We address this gap by proposing a comprehensive definition of trustworthiness inspired by human communication theory (Grice, 1975). Our definition identifies six orthogonal trust dimensions that provide interpretable measures of trustworthiness. Building on this definition, we introduce the Attention Trust Score (A -Trust), a lightweight, attention-based method for evaluating the trustworthiness of messages. We then develop a principled trust management system (TMS) for LLM -MAS that supports both message-level and agent-level trust assessments. Experiments across diverse multi-agent settings and tasks demonstrate that our TMS significantly improves robustness against malicious inputs.
Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 34 of its 56 references that are in the map. First posted on arXiv 3 Jun 2025.
Builds on
34 of 56 references.
Closest ancestors · 5
Same direction, newest first
- 2025AgentAuditor: Human-Level Safety and Security Evaluation for LLM Agents31 May
- 2025Red-Teaming LLM Multi-Agent Systems via Communication Attacks20 Feb
- 2024Breaking Agents: Compromising Autonomous LLM Agents Through Malfunction Amplification30 Jul
- 2024Flooding Spread of Manipulated Knowledge in LLM-Based Multi-Agent Communities10 Jul
- 2024PsySafe: A Comprehensive Framework for Psychological-based Attack, Defense, and Evaluation of Multi-agent System Safety22 Jan
Other references · 29
- 2020Measuring Massive Multitask Language Understandingcited by 7,963
- 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
- 2023Let's Verify Step by Stepcited by 3,789
- 2021Program Synthesis with Large Language Modelscited by 3,637
- 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,988
- 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,808
- 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 1,395
- 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,375
- 2022In-context Learning and Induction Headscited by 964
- 2020StereoSet: Measuring stereotypical bias in pretrained language modelscited by 924
- 2023CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquingcited by 783
- 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 743
17 more
- 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Modelscited by 651
- 2025Multi-Agent Collaboration Mechanisms: A Survey of LLMscited by 431
- 2023MedAgents: Large Language Models as Collaborators for Zero-shot Medical Reasoningcited by 316
- 2023S3: Social-network Simulation System with Large Language Model-Empowered Agentscited by 233
- 2023AI Control: Improving Safety Despite Intentional Subversioncited by 229
- 2024Large Language Model-Based Agents for Software Engineering: A Surveycited by 178
- 2023Language Agents with Reinforcement Learning for Strategic Play in the Werewolf Gamecited by 152
- 2024AgentScope: A Flexible yet Robust Multi-Agent Platformcited by 121
- 2023Shepherd: A Critic for Language Model Generationcited by 97
- 2023Successor Heads: Recurring, Interpretable Attention Heads In The Wildcited by 77
- 2024MultiAgent Collaboration Attack: Investigating Adversarial Attacks in Large Language Model Collaborations via Debatecited by 56
- 2024MACM: Utilizing a Multi-Agent System for Condition Mining in Solving Complex Mathematical Problemscited by 41
- 2024Towards Knowledge Checking in Retrieval-augmented Generation: A Representation Perspectivecited by 20
- 2024TrustScore: Reference-Free Evaluation of LLM Response Trustworthinesscited by 19
- 2024Are LLMs Good Zero-Shot Fallacy Classifiers?cited by 12
- 2024Evaluation of an LLM in Identifying Logical Fallacies: A Call for Rigor When Adopting LLMs in HCI Researchcited by 10
- 2024Make LLMs better zero-shot reasoners: Structure-orientated autonomous reasoningcited by 6
Built on by
Citing papers per year: 2025: 4; 2026: 2.
6 papers.
- 2025A Survey of LLM-Driven AI Agent Communication: Protocols, Security Risks, and Defense Countermeasurescited by 57
- 2025BlindGuard: Safeguarding LLM-based Multi-Agent Systems under Unknown Attackscited by 38
- 2025Metacognitive Self-Correction for Multi-Agent System via Prototype-Guided Next-Execution Reconstructioncited by 7
- 2026Don't Trust Stubborn Neighbors: A Security Framework for Agentic Networkscited by 4
- 2025The AI in the Mirror: LLM Self-Recognition in an Iterated Public Goods Gamecited by 1
- 2026SoK: When Safe Agents Fail Together: The Security of Multi Agent LLM Systemscited by 1
- 2026SoK: When Safe Agents Fail Together: The Security of Multi Agent LLM Systems1 Sep 2026
- 2026Don't Trust Stubborn Neighbors: A Security Framework for Agentic Networks16 Mar 2026
- 2025Metacognitive Self-Correction for Multi-Agent System via Prototype-Guided Next-Execution Reconstruction16 Oct 2025
- 2025The AI in the Mirror: LLM Self-Recognition in an Iterated Public Goods Game25 Aug 2025
- 2025BlindGuard: Safeguarding LLM-based Multi-Agent Systems under Unknown Attacks11 Aug 2025
- 2025A Survey of LLM-Driven AI Agent Communication: Protocols, Security Risks, and Defense Countermeasures24 Jun 2025
Similar, not cited
Close in wording, no citation either way
- 2025A Survey on Trustworthy LLM Agents: Threats and Countermeasures
- 2025The Trust Paradox in LLM-Based Multi-Agent Systems: When Collaboration Becomes a Security Vulnerability
- 2025TRiSM for Agentic AI: A Review of Trust, Risk, and Security Management in LLM-based Agentic Multi-Agent Systems
- 2026Towards trustworthy agentic AI: a comprehensive survey of safety, robustness, privacy, and system security
- 2024Can We Trust AI Agents? A Case Study of an LLM-Based Multi-Agent System for Ethical AI
- 2025Comprehensive Vulnerability Analysis is Necessary for Trustworthy LLM-MAS
- 2026Trustworthy Agent Network: Trust in Agent Networks Must Be Baked In, Not Bolted On
- 2026A Unified Evaluation Framework for Trustworthy Large Language Models, Agentic AI, and Multimodal Systems