mlsanityData to 7 Oct 2026

TRiSM for Agentic AI: A Review of Trust, Risk, and Security Management in LLM-based Agentic Multi-Agent Systems

arXiv 2506.04133 ↗First posted 4 Jun 2025cs.AIcited by 58

On the map: LLM agents and tool use → Agent threats, guardrails and governance

Abstract (arXiv)

Agentic AI systems, built upon large language models (LLMs) and deployed in multi-agent configurations, are redefining intelligence, autonomy, collaboration, and decision-making across enterprise and societal domains. This review presents a structured analysis of Trust, Risk, and Security Management (TRiSM) in the context of LLM-based Agentic Multi-Agent Systems (AMAS). We begin by examining the conceptual foundations of Agentic AI and highlight its architectural distinctions from traditional AI agents. We then adapt and extend the AI TRiSM framework for Agentic AI, structured around key pillars: textit Explainability, ModelOps, Security, Privacy and textittheir Lifecycle Governance, each contextualized to the challenges of AMAS. A risk taxonomy is proposed to capture the unique threats and vulnerabilities of Agentic AI, ranging from coordination failures to prompt-based adversarial manipulation. To support practical assessment in Agentic AI works, we introduce two novel metrics: the Component Synergy Score (CSS), which quantifies the quality of inter-agent collaboration, and the Tool Utilization Efficacy (TUE), which evaluates the efficiency of tool use within agent workflows. We further discuss strategies for improving explainability in Agentic AI, as well as approaches to enhancing security and privacy through encryption, adversarial robustness, and regulatory compliance. The review concludes with a research roadmap for the responsible development and deployment of Agentic AI, highlighting key directions to align emerging systems with TRiSM principles-ensuring safety, transparency, and accountability in their operation.

Key facts. Cited by 58 papers in the map (data through 7 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 89 of its 211 references that are in the map. First posted on arXiv 4 Jun 2025.

Builds on

89 of 211 references.

Closest ancestors · 8

Same direction, newest first

  1. 2025Seven Security Challenges in Cross-domain Multi-agent LLM Systems28 May
  2. 2025An Outlook on the Opportunities and Challenges of Multi-Agent AI Systems23 May
  3. 2025Automating Security Audit Using Large Language Model based Agent: An Exploration Experiment15 May
  4. 2025Security of Internet of Agents: Attacks and Countermeasures12 May
  5. 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deployment22 Apr
  6. 2025A Survey on Trustworthy LLM Agents: Threats and Countermeasures12 Mar
  7. 2024Prompt Infection: LLM-to-LLM Prompt Injection within Multi-Agent Systems9 Oct
  8. 2024Breaking Agents: Compromising Autonomous LLM Agents Through Malfunction Amplification30 Jul

Other references · 81

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  2. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  3. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  4. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  5. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  6. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  7. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
  8. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
  9. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,906
  10. 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,761
  11. 2019Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AIcited by 1,543
  12. 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 1,348
38 more
  1. 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,173
  2. 2023ChatDev: Communicative Agents for Software Developmentcited by 893
  3. 2022Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learnedcited by 702
  4. 2023AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviorscited by 580
  5. 2024WebVoyager: Building an End-to-End Web Agent with Large Multimodal Modelscited by 424
  6. 2025Multi-Agent Collaboration Mechanisms: A Survey of LLMscited by 424
  7. 2025The AI Scientist-v2: Workshop-Level Automated Scientific Discovery via Agentic Tree Searchcited by 375
  8. 2023Auto-GPT for Online Decision Making: Benchmarks and Additional Opinionscited by 250
  9. 2025Large Language Model Agent: A Survey on Methodology, Applications and Challengescited by 194
  10. 2025AI Agents vs. Agentic AI: A Conceptual Taxonomy, Applications and Challengescited by 167
  11. 2017A Survey on Homomorphic Encryption Schemes: Theory and Implementationcited by 154
  12. 2022MRKL Systems: A modular, neuro-symbolic architecture that combines large language models, external knowledge sources and discrete reasoningcited by 153
  13. 2023OpenAgents: An Open Platform for Language Agents in the Wildcited by 136
  14. 2022OpenXAI: Towards a Transparent Evaluation of Model Explanationscited by 125
  15. 2024AI Agents Under Threat: A Survey of Key Security Challenges and Future Pathwayscited by 122
  16. 2024GLoRe: When, Where, and How to Improve LLM Reasoning via Global and Local Refinementscited by 101
  17. 2023Can Large Language Models Explain Themselves? A Study of LLM-Generated Self-Explanationscited by 90
  18. 2024SciAgents: Automating scientific discovery through multi-agent intelligent graph reasoningcited by 90
  19. 2025A Survey of LLM-based Agents in Medicine: How far are we from Baymax?cited by 84
  20. 2024Agent Planning with World Knowledge Modelcited by 66
  21. 2025Open Challenges in Multi-Agent Security: Towards Secure Systems of Interacting AI Agentscited by 66
  22. 2025Authenticated Delegation and Authorized AI Agentscited by 63
  23. 2025Beyond Self-Talk: A Communication-Centric Survey of LLM-Based Multi-Agent Systemscited by 58
  24. 2024All Languages Matter: Evaluating LMMs on Culturally Diverse 100 Languagescited by 55
  25. 2024A Survey on LLM-based Multi-Agent System: Recent Advances and New Frontiers in Applicationcited by 54
  26. 2018Interpretable to Whom? A Role-based Model for Analyzing Interpretable Machine Learning Systemscited by 47
  27. 2024ALI-Agent: Assessing LLMs' Alignment with Human Values via Agent-based Evaluationcited by 46
  28. 2024MegaAgent: A Large-Scale Autonomous LLM-based Multi-Agent System Without Predefined SOPscited by 45
  29. 2023A Perspective on Explainable Artificial Intelligence Methods: SHAP and LIMEcited by 42
  30. 2025LLMs for Explainable AI: A Comprehensive Surveycited by 42
  31. 2025Multi-agent Embodied AI: Advances and Future Directionscited by 36
  32. 2019Explainability in Human-Agent Systemscited by 34
  33. 2025Generative to Agentic AI: Survey, Conceptualization, and Challengescited by 32
  34. 2025LLM-Based Human-Agent Collaboration and Interaction Systems: A Surveycited by 28
  35. 2024Agent AI with LangGraph: A Modular Framework for Enhancing Machine Translation Using Large Language Modelscited by 27
  36. 2025Position: Uncertainty Quantification Needs Reassessment for Large-language Model Agentscited by 27
  37. 2017Calendar.help: Designing a Workflow-Based Scheduling Agent with Humans in the Loopcited by 26
  38. 2025Among Us: A Sandbox for Measuring and Detecting Agentic Deceptioncited by 24

50 most cited of 81 All 81 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 25; 2026: 33.

58 papers.

Sort:

By area:

  1. 2025A Survey of Context Engineering for Large Language Modelscited by 120
  2. 2025Agentic AI: A Comprehensive Survey of Architectures, Applications, and Future Directionscited by 46
  3. 2025Agentic AI Security: Threats, Defenses, Evaluation, and Open Challengescited by 26
  4. 2025A Survey on Agentic Multimodal Large Language Modelscited by 19
  5. 2025AI Agents with Decentralized Identifiers and Verifiable Credentialscited by 17
  6. 2025AgentGuard: Runtime Verification of AI Agentscited by 16
  7. 2025Systematization of Knowledge: Security and Safety in the Model Context Protocol Ecosystemcited by 14
  8. 2025UAVs Meet Agentic AI: A Multidomain Survey of Autonomous Aerial Intelligence and Agentic UAVscited by 13
  9. 2025AGENTSAFE: A Unified Framework for Ethical Assurance and Governance in Agentic AIcited by 9
  10. 2026A Systematic Survey of Security Threats and Defenses in LLM-Based AI Agents: A Layered Attack Surface Frameworkcited by 9
  11. 2026The Why Behind the Action: Unveiling Internal Drivers via Agentic Attributioncited by 8
  12. 2025Bridging AI and Software Security: A Comparative Vulnerability Assessment of LLM Agent Deployment Paradigmscited by 7
38 more
  1. 2025BetaWeb: Towards a Blockchain-enabled Trustworthy Agentic Webcited by 7
  2. 2025Federation of Agents: A Semantics-Aware Communication Fabric for Large-Scale Agentic AIcited by 6
  3. 2026Autonomous Action Runtime Management(AARM):A System Specification for Securing AI-Driven Actions at Runtimecited by 5
  4. 2026Toward Secure LLM Agents: Threat Surfaces, Attacks, Defenses, and Evaluationcited by 5
  5. 2025The Dark Side of LLMs: Agent-based Attack Vectors for System-level Compromisecited by 4
  6. 2025With Great Capabilities Come Great Responsibilities: Introducing the Agentic Risk & Capability Framework for Governing Agentic AI Systemscited by 4
  7. 2026AI Organizations are More Effective but Less Aligned than Individual Agentscited by 4
  8. 2026Do Coding Agents Understand Least-Privilege Authorization?cited by 4
  9. 2025AdvEvo-MARL: Shaping Internalized Safety through Adversarial Co-Evolution in Multi-Agent Reinforcement Learningcited by 3
  10. 2025Agentic Satellite-Augmented Low-Altitude Economy and Terrestrial Networks: A Survey on Generative Approachescited by 2
  11. 2025Hybrid Agentic AI and Multi-Agent Systems in Smart Manufacturingcited by 2
  12. 2025The Evolution of Agentic AI in Cybersecurity: From Single LLM Reasoners to Multi-Agent Systems and Autonomous Pipelinescited by 2
  13. 2026Human Society-Inspired Approaches to Agentic AI Security: The 4C Frameworkcited by 2
  14. 2026Beyond Fluency: Toward Reliable Trajectories in Agentic IRcited by 2
  15. 2026GAMMAF: A Common Framework for Graph-Based Anomaly Monitoring Benchmarking in LLM Multi-Agent Systemscited by 2
  16. 2026When Child Inherits: Modeling and Exploiting Subagent Spawn in Multi-Agent Networkscited by 2
  17. 2026Language model agents show in-group trust bias invisible to standard behavioural auditscited by 2
  18. 2025Cisco Integrated AI Security and Safety Framework Reportcited by 1
  19. 2025Agentic Persona Control and Task State Tracking for Realistic User Simulation in Interactive Scenarioscited by 1
  20. 2026Context Kubernetes: Declarative Orchestration of Enterprise Knowledge for Agentic AI Systemscited by 1
  21. 2026Beyond the 'Diff': Addressing Agentic Entropy in Agentic Software Developmentcited by 1
  22. 2026The Containment Gap: How Deployed Agentic AI Frameworks Fail Public-Facing Safety Requirementscited by 1
  23. 2026Adversarial Attacks in Multi-Agent LLM Pipelines: Unveiling Structural Vulnerabilities in Agentic AI Architecturescited by 1
  24. 2026SoK: When Safe Agents Fail Together: The Security of Multi Agent LLM Systemscited by 1
  25. 2025Plural Voices, Single Agent: Towards Inclusive AI in Multi-User Domestic Spacescited by 0
  26. 2025Foundations of Artificial Intelligence Frameworks: Notion and Limits of AGIcited by 0
  27. 2025Enhancing Trustworthiness with Mixed Precision: Benchmarks, Opportunities, and Challengescited by 0
  28. 2025MALCDF: A Distributed Multi-Agent LLM Framework for Real-Time Cybercited by 0
  29. 2025Adaptive Accountability in Networked MAS: Tracing and Mitigating Emergent Norms at Scalecited by 0
  30. 2026Intellicise Wireless Networks Meet Agentic AI: A Security and Privacy Perspectivecited by 0
  31. 2026Performance Comparison of IBN orchestration using LLM and SLMscited by 0
  32. 2026Fairness in Multi-Agent Systems for Software Engineering: An SDLC-Oriented Rapid Reviewcited by 0
  33. 2026Executor-Side Progressive Risk-Gated Actuation for Agentic AI in Wireless Supervisory Controlcited by 0
  34. 2026Agentic AI Ecosystems in Higher Education: A Perspective on AI Agents to Emerging Inclusive, Agentic Multi-Agent AI Framework for Learning, Teaching and Institutional Intelligencecited by 0
  35. 2026Why Trust Your Agent? Empirical Security Gains from TRiSM-Guided Agentic Workflows in Healthcarecited by 0
  36. 2026Memory as an Attack Surface in LLM Agents: A Study on Multiple-Choice Question Answeringcited by 0
  37. 2026A Diagnostic Framework for AI Agent Behaviorcited by 0
  38. 2026From Monoliths to Swarms: A Study of Attack Surface Evolution in the Transition to Multi-Agent Web Systemscited by 0

50 most cited of 58 All 58 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Towards trustworthy agentic AI: a comprehensive survey of safety, robustness, privacy, and system security
  2. 2026Trustworthy Agentic AI: A Comprehensive Cybersecurity and Systems Survey on Threat Landscapes, Defense Architectures, and Open Challenges
  3. 2026SoK: The Attack Surface of Agentic AI - Tools and Autonomy
  4. 2026The Attack and Defense Landscape of Agentic AI: A Comprehensive Survey
  5. 2026LLM-Based Agents for Software and Systems Security: Approaches, Applications, and Assessment
  6. 2025Risk Analysis Techniques for Governed LLM-based Multi-Agent Systems
  7. 2025Breaking Agent Backbones: Evaluating the Security of Backbone LLMs in AI Agents
  8. 2025A Survey of LLM-Driven AI Agent Communication: Protocols, Security Risks, and Defense Countermeasures