mlsanityData to 14 Oct 2026

Trustworthy Agentic AI: A Comprehensive Cybersecurity and Systems Survey on Threat Landscapes, Defense Architectures, and Open Challenges

arXiv 2609.13731 ↗First posted 12 Sep 2026cs.AIcited by 0

On the map: LLM agents and tool use → Agent threats, guardrails and governance

Abstract (arXiv)

The transition from passive foundation models to autonomous, goal-directed agentic AI systems has introduced unprecedented capabilities by coupling recursive cognitive reasoning loops, persistent memory architectures, live tool execution planes, and multi-agent collaboration topologies. However, granting probabilistic neural cores execution authority across filesystems, networks, and cloud infrastructure dissolves classical security perimeters: natural language simultaneously serves as input data, internal control code, and communication protocols, exposing a Turing-complete blast radius where untrusted data represents executable instructions. This survey delivers a comprehensive systems-security reference framework for trustworthy agentic AI, synthesizing 206 foundational studies and regulatory standards. We formalize the general agent architecture as a stateful 5-tuple and establish a 6-dimensional trustworthiness taxonomy covering security, safety, privacy, explainability, fairness, and accountability. We systematically analyze threat surfaces across intra-execution loops and interaction planes, formulate a multi-layered zero-trust defense-in-depth architecture integrating Dual-LLM isolation, Capability-Based Access Control, kernel eBPF probes, and sandboxed runtimes, review standardized evaluation benchmarks, and map technical controls to international AI governance frameworks.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 92 of its 193 references that are in the map. First posted on arXiv 12 Sep 2026.

Builds on

92 of 193 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents5 Mar
  2. 2024Agent Smith: A Single Image Can Jailbreak One Million Multimodal LLM Agents Exponentially Fast13 Feb
  3. 2024R-Judge: Benchmarking Safety Risk Awareness for LLM Agents18 Jan
  4. 2024Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training10 Jan

Other references · 88

  1. 2017Attention Is All You Needcited by 60,361
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2022Training language models to follow instructions with human feedbackcited by 15,971
  4. 2024The Llama 3 Herd of Modelscited by 14,715
  5. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  6. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
  7. 2017Graph Attention Networkscited by 8,923
  8. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  9. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  10. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  11. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  12. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
38 more
  1. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  2. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  3. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  4. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  5. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,568
  6. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
  7. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
  8. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  9. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
  10. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,998
  11. 2016Deep Learning with Differential Privacycited by 2,728
  12. 2017Billion-scale similarity search with GPUscited by 2,497
  13. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,348
  14. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,281
  15. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,242
  16. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,191
  17. 2020Extracting Training Data from Large Language Modelscited by 2,189
  18. 2022Survey of Hallucination in Natural Language Generationcited by 2,075
  19. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,988
  20. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,947
  21. 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,808
  22. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,745
  23. 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,737
  24. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,651
  25. 2016Concrete Problems in AI Safetycited by 1,642
  26. 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,451
  27. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,443
  28. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,439
  29. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,420
  30. 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 1,395
  31. 2023MemGPT: Towards LLMs as Operating Systemscited by 1,389
  32. 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,332
  33. 2023AgentBench: Evaluating LLMs as Agentscited by 1,267
  34. 2023Mind2Web: Towards a Generalist Agent for the Webcited by 1,242
  35. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,207
  36. 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,190
  37. 2023GAIA: a benchmark for General AI Assistantscited by 1,128
  38. 2022Red Teaming Language Models with Language Modelscited by 1,026

50 most cited of 88 All 88 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Towards trustworthy agentic AI: a comprehensive survey of safety, robustness, privacy, and system security
  2. 2026The Attack and Defense Landscape of Agentic AI: A Comprehensive Survey
  3. 2026AgentTrust: A Self-Improving Trust Layer for AI-Agent Actions
  4. 2026Trustworthy Agent Network: Trust in Agent Networks Must Be Baked In, Not Bolted On
  5. 2025Agentic AI Security: Threats, Defenses, Evaluation, and Open Challenges
  6. 2026Trustworthy Agentic AI Requires Deterministic Architectural Boundaries
  7. 2025Systems Security Foundations for Agentic Computing
  8. 2026When Agents Handle Secrets: A Survey of Confidential Computing for Agentic AI