mlsanityData to 14 Oct 2026

To trust or not to trust: Attention-based Trust Management for LLM Multi-Agent Systems

arXiv 2506.02546 ↗First posted 3 Jun 2025cs.CRcited by 6

On the map: LLM agents and tool use → Agent threats, guardrails and governance

Abstract (arXiv)

Large Language Model-based Multi-Agent Systems (LLM-MAS) have demonstrated strong capabilities in solving complex tasks but remain vulnerable when agents receive unreliable messages. This vulnerability stems from a fundamental gap: LLM agents treat all incoming messages equally without evaluating their trustworthiness. While some existing studies approach trustworthiness, they focus on a single type of harmfulness rather than analyze it in a holistic approach from multiple trustworthiness perspectives. We address this gap by proposing a comprehensive definition of trustworthiness inspired by human communication theory (Grice, 1975). Our definition identifies six orthogonal trust dimensions that provide interpretable measures of trustworthiness. Building on this definition, we introduce the Attention Trust Score (A -Trust), a lightweight, attention-based method for evaluating the trustworthiness of messages. We then develop a principled trust management system (TMS) for LLM -MAS that supports both message-level and agent-level trust assessments. Experiments across diverse multi-agent settings and tasks demonstrate that our TMS significantly improves robustness against malicious inputs.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 34 of its 56 references that are in the map. First posted on arXiv 3 Jun 2025.

Builds on

34 of 56 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025AgentAuditor: Human-Level Safety and Security Evaluation for LLM Agents31 May
  2. 2025Red-Teaming LLM Multi-Agent Systems via Communication Attacks20 Feb
  3. 2024Breaking Agents: Compromising Autonomous LLM Agents Through Malfunction Amplification30 Jul
  4. 2024Flooding Spread of Manipulated Knowledge in LLM-Based Multi-Agent Communities10 Jul
  5. 2024PsySafe: A Comprehensive Framework for Psychological-based Attack, Defense, and Evaluation of Multi-agent System Safety22 Jan

Other references · 29

  1. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  2. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  3. 2023Let's Verify Step by Stepcited by 3,789
  4. 2021Program Synthesis with Large Language Modelscited by 3,637
  5. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,988
  6. 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,808
  7. 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 1,395
  8. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,375
  9. 2022In-context Learning and Induction Headscited by 964
  10. 2020StereoSet: Measuring stereotypical bias in pretrained language modelscited by 924
  11. 2023CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquingcited by 783
  12. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 743
17 more
  1. 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Modelscited by 651
  2. 2025Multi-Agent Collaboration Mechanisms: A Survey of LLMscited by 431
  3. 2023MedAgents: Large Language Models as Collaborators for Zero-shot Medical Reasoningcited by 316
  4. 2023S3: Social-network Simulation System with Large Language Model-Empowered Agentscited by 233
  5. 2023AI Control: Improving Safety Despite Intentional Subversioncited by 229
  6. 2024Large Language Model-Based Agents for Software Engineering: A Surveycited by 178
  7. 2023Language Agents with Reinforcement Learning for Strategic Play in the Werewolf Gamecited by 152
  8. 2024AgentScope: A Flexible yet Robust Multi-Agent Platformcited by 121
  9. 2023Shepherd: A Critic for Language Model Generationcited by 97
  10. 2023Successor Heads: Recurring, Interpretable Attention Heads In The Wildcited by 77
  11. 2024MultiAgent Collaboration Attack: Investigating Adversarial Attacks in Large Language Model Collaborations via Debatecited by 56
  12. 2024MACM: Utilizing a Multi-Agent System for Condition Mining in Solving Complex Mathematical Problemscited by 41
  13. 2024Towards Knowledge Checking in Retrieval-augmented Generation: A Representation Perspectivecited by 20
  14. 2024TrustScore: Reference-Free Evaluation of LLM Response Trustworthinesscited by 19
  15. 2024Are LLMs Good Zero-Shot Fallacy Classifiers?cited by 12
  16. 2024Evaluation of an LLM in Identifying Logical Fallacies: A Call for Rigor When Adopting LLMs in HCI Researchcited by 10
  17. 2024Make LLMs better zero-shot reasoners: Structure-orientated autonomous reasoningcited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 2.

6 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025A Survey on Trustworthy LLM Agents: Threats and Countermeasures
  2. 2025The Trust Paradox in LLM-Based Multi-Agent Systems: When Collaboration Becomes a Security Vulnerability
  3. 2025TRiSM for Agentic AI: A Review of Trust, Risk, and Security Management in LLM-based Agentic Multi-Agent Systems
  4. 2026Towards trustworthy agentic AI: a comprehensive survey of safety, robustness, privacy, and system security
  5. 2024Can We Trust AI Agents? A Case Study of an LLM-Based Multi-Agent System for Ethical AI
  6. 2025Comprehensive Vulnerability Analysis is Necessary for Trustworthy LLM-MAS
  7. 2026Trustworthy Agent Network: Trust in Agent Networks Must Be Baked In, Not Bolted On
  8. 2026A Unified Evaluation Framework for Trustworthy Large Language Models, Agentic AI, and Multimodal Systems