mlsanityData to 14 Oct 2026

Why Trust Your Agent? Empirical Security Gains from TRiSM-Guided Agentic Workflows in Healthcare

arXiv 2606.28666 ↗First posted 27 Jun 2026cs.CR cs.AIcited by 0

On the map: LLM agents and tool use → Agent threats, guardrails and governance

Abstract (arXiv)

Agent-based AI has enabled the automation of tasks by exposing application tools and resources to large language models (LLMs). However, to improve scope and accuracy, agents are often given access rights that exceed those of ordinary users, introducing significant security risks. AI is routinely integrated into applications with a disregard to security, risking data exposure and breaching regulations. This paper applies the AI Trust, Risk, and Security Management (TRiSM) framework to a medical report-generation application to demonstrate how an insecure agent workflow can be transformed into security-conscious agentic workflow. Both workflows were evaluated across five LLMs (Claude Haiku 4.5, GPT-4.1-nano, GPT-4.1-mini, GPT-5.4-mini, and Gemini 2.5 Flash) on two report types, totalling 800 generations and 500 attack scenarios including RAG poisoning, data-field injection, and client-side network injection. The TRiSM-guided agentic workflow reduced mean attack success rates from 31% to 10% for RAG poisoning and from 42% to 25% for data-field injection, while eliminating the network injection vector entirely through server-side prompt construction. Furthermore, report accuracy increased by 14 percentage points (72.5% to 86.5%) with the agentic workflow, demonstrating a secure design which provides more reliable outputs. This paper contributes to knowledge by demonstrating least-privilege, defence in depth agentic workflows improving security and accuracy, while also highlighting model choice is a necessary architectural consideration.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 24 of its 35 references that are in the map. First posted on arXiv 27 Jun 2026.

Builds on

24 of 35 references.

Closest ancestors · 7

Same direction, newest first

  1. 2026Memory poisoning and secure multi-agent systems20 Mar
  2. 2026The 2025 AI Agent Index: Documenting Technical and Safety Features of Deployed Agentic AI Systems19 Feb
  3. 2025Is Vibe Coding Safe? Benchmarking Vulnerability of Agent-Generated Code in Real-World Tasks2 Dec
  4. 2025Securing AI Agents Against Prompt Injection Attacks19 Nov
  5. 2025TRiSM for Agentic AI: A Review of Trust, Risk, and Security Management in LLM-based Agentic Multi-Agent Systems4 Jun
  6. 2025Commercial LLM Agents Are Already Vulnerable to Simple Yet Dangerous Attacks12 Feb
  7. 2024AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Bases17 Jul

Other references · 17

  1. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,443
  2. 2019Deep Leakage from Gradientscited by 1,021
  3. 2023Prompt Injection attack against LLM-integrated Applicationscited by 605
  4. 2025Model Context Protocol (MCP): Landscape, Security Threats, and Future Research Directionscited by 265
  5. 2025AI Agents vs. Agentic AI: A Conceptual Taxonomy, Applications and Challengescited by 169
  6. 2024Optimization-based Prompt Injection Attack to LLM-as-a-Judgecited by 143
  7. 2024AI Agents Under Threat: A Survey of Key Security Challenges and Future Pathwayscited by 127
  8. 2024LLM Dataset Inference: Did you train on my dataset?cited by 110
  9. 2023Language Model Inversioncited by 68
  10. 2025Securing the Model Context Protocol (MCP): Risks, Controls, and Governancecited by 23
  11. 2024Security Threats in Agentic AI Systemcited by 11
  12. 2020Evaluation of Inference Attack Models for Deep Learning on Medical Datacited by 9
5 more
  1. 2025Your RAG is Unfair: Exposing Fairness Vulnerabilities in Retrieval-Augmented Generation via Backdoor Attackscited by 8
  2. 2025Operationalizing Data Minimization for Privacy-Preserving LLM Promptingcited by 8
  3. 2026MPIB: A Benchmark for Medical Prompt Injection Attacks and Clinical Safety in LLMscited by 3
  4. 2025Analysing Environmental Efficiency in AI for X-Ray Diagnosiscited by 1
  5. 2025AI TIPS 2.0: A Comprehensive Framework for Operationalizing AI Governancecited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026LLM-Based Agents for Software and Systems Security: Approaches, Applications, and Assessment
  2. 2024Can We Trust AI Agents? A Case Study of an LLM-Based Multi-Agent System for Ethical AI
  3. 2025A Field Guide to Deploying AI Agents in Clinical Practice
  4. 2026Trustworthy Agentic AI: Failure Modes, Mitigation Strategies, and a Lifecycle Framework for Autonomous LLM Systems
  5. 2024Whispers in the Machine: Confidentiality in Agentic Systems
  6. 2026Agentic AI Enhances Physician Trust in Clinical Decision Making
  7. 2026AgentTrap: Measuring Runtime Trust Failures in Third-Party Agent Skills
  8. 2024Agentic LLM Workflows for Generating Patient-Friendly Medical Reports