mlsanityData to 7 Oct 2026

Isolation as a First-Class Principle for LLM-Agent System Safety: Concepts, Taxonomy, Challenges and Future Directions

arXiv 2607.12406 ↗First posted 14 Jul 2026cs.AIcited by 1

On the map: LLM agents and tool use → Agent threats, guardrails and governance

Abstract (arXiv)

The capability of LLM agents to function as the ``brain'' of a system fundamentally expands the scope of analysis beyond a standalone model. Consequently, safety is no longer only about input--output content alignment. It also concerns system behavior and real-world execution outcomes. However, the current literature is fragmented across attack types, applications, and benchmarks. This makes it hard to explain why failures such as prompt injection, tool misuse, and memory poisoning often share the same structural cause, and how they spread through an agent workflow. In this survey, we treat isolation as a first-class principle for LLM-agent system safety. By isolation, we refer to the separation of user inputs, tool access, execution channels, inter-agent communication, and environment-originated context. We organize the literature with a boundary-centric taxonomy of five boundaries: user-agent, agent-tool, agent-execution, agent-agent, and system-environment. This view helps identify where the loss of isolation first occurs, how compromise propagates across boundaries, and which defenses are most relevant at each interface. We also summarize cross-boundary failure paths, discuss open challenges, and outline a research agenda for isolation-by-construction in future agent systems.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 112 of its 155 references that are in the map. First posted on arXiv 14 Jul 2026.

Builds on

112 of 155 references.

Closest ancestors · 45

Same direction, newest first

  1. 2026MOSAIC-Bench: Measuring Compositional Vulnerability Induction in Coding Agents5 May
  2. 2026MEMSAD: Gradient-Coupled Anomaly Detection for Memory Poisoning in Retrieval-Augmented Agents5 May
  3. 2026Trojan Hippo: Weaponizing Agent Memory for Data Exfiltration3 May
  4. 2026AgentVisor: Defending LLM Agents Against Prompt Injection via Semantic Virtualization27 Apr
  5. 2026Your Agent is More Brittle Than You Think: Uncovering Indirect Injection Vulnerabilities in Agentic LLMs4 Apr
  6. 2026A Framework for Formalizing LLM Agent Security19 Mar
  7. 2026Caging the Agents: A Zero Trust Security Architecture for Autonomous AI in Healthcare18 Mar
  8. 2026The Attack and Defense Landscape of Agentic AI: A Comprehensive Survey11 Mar
  9. 2026AttriGuard: Defeating Indirect Prompt Injection in LLM Agents via Causal Attribution of Tool Invocations11 Mar
  10. 2026AgentSentry: Mitigating Indirect Prompt Injection in LLM Agents via Temporal Causal Diagnostics and Context Purification26 Feb
  11. 2026AdapTools: Adaptive Tool-based Indirect Prompt Injection Attacks on Agentic LLMs24 Feb
  12. 2026ToolSafe: Enhancing Tool Invocation Safety of LLM-based agents via Proactive Step-level Guardrail and Feedback15 Jan
33 more
  1. 2025MCP Security Bench (MSB): Benchmarking Attacks Against Model Context Protocol in LLM Agents14 Oct
  2. 2025A Multi-Agent LLM Defense Pipeline Against Prompt Injection Attacks16 Sep
  3. 2025VPI-Bench: Visual Prompt Injection Attacks for Computer-Use Agents3 Jun
  4. 2025MPMA: Preference Manipulation Attack Against Model Context Protocol16 May
  5. 2025Prompt Injection Attack to Tool Selection in LLM Agents28 Apr
  6. 2025WASP: Benchmarking Web Agent Security Against Prompt Injection Attacks22 Apr
  7. 2025Advancing Embodied Agent Security: From Safety Benchmarks to Input Moderation22 Apr
  8. 2025The Obvious Invisible Threat: LLM-Powered GUI Agents' Vulnerability to Fine-Print Injections15 Apr
  9. 2025ShieldAgent: Shielding Agents via Verifiable Safety Policy Reasoning26 Mar
  10. 2025SafeArena: Evaluating the Safety of Autonomous Web Agents6 Mar
  11. 2025AgentSafe: Safeguarding Large Language Model-based Multi-agent Systems via Hierarchical Data Management6 Mar
  12. 2025Why Are Web AI Agents More Vulnerable Than Standalone LLMs? A Security Analysis27 Feb
  13. 2025Red-Teaming LLM Multi-Agent Systems via Communication Attacks20 Feb
  14. 2025AgentGuard: Repurposing Agentic Orchestrator for Safety Evaluation of Tool Orchestration13 Feb
  15. 2024The Task Shield: Enforcing Task Alignment to Defend Against Indirect Prompt Injection in LLM Agents21 Dec
  16. 2024From Allies to Adversaries: Manipulating LLM Tool-Calling through Adversarial Injection13 Dec
  17. 2024RedCode: Risky Code Execution and Generation Benchmark for Code Agents12 Nov
  18. 2024AdvAgent: Controllable Blackbox Red-teaming on Web Agents22 Oct
  19. 2024Imprompter: Tricking LLM Agents into Improper Tool Use19 Oct
  20. 2024Refusal-Trained LLMs Are Easily Jailbroken As Browser Agents11 Oct
  21. 2024FATS: A Prompt Injection Attack Utilizing Feign Security Agents with Deceptive Few-shots Learning11 Oct
  22. 2024ST-WebAgentBench: A Benchmark for Evaluating Safety and Trustworthiness in Web Agents9 Oct
  23. 2024EIA: Environmental Injection Attack on Generalist Web Agents for Privacy Leakage17 Sep
  24. 2024Compromising Embodied Agents with Contextual Backdoor Attacks6 Aug
  25. 2024Breaking Agents: Compromising Autonomous LLM Agents Through Malfunction Amplification30 Jul
  26. 2024AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Bases17 Jul
  27. 2024Flooding Spread of Manipulated Knowledge in LLM-Based Multi-Agent Communities10 Jul
  28. 2024BadAgent: Inserting and Activating Backdoor Attacks in LLM Agents5 Jun
  29. 2024Can We Trust Embodied Agents? Exploring Backdoor Attacks against Embodied LLM-based Decision-Making Systems27 May
  30. 2024LLM Agents can Autonomously Exploit One-day Vulnerabilities11 Apr
  31. 2024AutoDefense: Multi-Agent LLM Defense against Jailbreak Attacks2 Mar
  32. 2024WIPI: A New Web Threat for LLM-Driven Web Agents26 Feb
  33. 2024LLM Agents can Autonomously Hack Websites6 Feb

Other references · 67

  1. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,339
  2. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
  3. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,326
  4. 2025A-MEM: Agentic Memory for LLM Agentscited by 1,000
  5. 2022Ignore Previous Prompt: Attack Techniques For Language Modelscited by 760
  6. 2023Prompt Injection attack against LLM-integrated Applicationscited by 568
  7. 2023Catastrophic Jailbreak of Open-source LLMs via Exploiting Generationcited by 439
  8. 2023SmoothLLM: Defending Large Language Models Against Jailbreaking Attackscited by 434
  9. 2023Detecting Language Model Attacks with Perplexitycited by 411
  10. 2023Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrationscited by 401
  11. 2023FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Promptscited by 368
  12. 2024Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attackcited by 364
38 more
  1. 2024StruQ: Defending Against Prompt Injection with Structured Queriescited by 325
  2. 2024Defending Against Indirect Prompt Injection Attacks With Spotlightingcited by 233
  3. 2024SORRY-Bench: Systematically Evaluating Large Language Model Safety Refusalcited by 224
  4. 2024Embodied Agent Interface: Benchmarking LLMs for Embodied Decision Makingcited by 142
  5. 2023Tensor Trust: Interpretable Prompt Injection Attacks from an Online Gamecited by 135
  6. 2024Tamper-Resistant Safeguards for Open-Weight LLMscited by 132
  7. 2023Abusing Images and Sounds for Indirect Instruction Injection in Multi-Modal LLMscited by 117
  8. 2024Eyes Closed, Safety On: Protecting Multimodal LLMs via Image-to-Text Transformationcited by 114
  9. 2023Jatmo: Prompt Injection Defense by Task-Specific Finetuningcited by 89
  10. 2024ToolSword: Unveiling Safety Issues of Large Language Models in Tool Learning Across Three Stagescited by 77
  11. 2024Defending Large Language Models Against Jailbreak Attacks via Layer-specific Editingcited by 77
  12. 2024Follow My Instruction and Spill the Beans: Scalable Data Extraction from Retrieval-Augmented Generation Systemscited by 76
  13. 2024Antidote: Post-fine-tuning Safety Alignment for Large Language Models against Harmful Fine-tuningcited by 71
  14. 2024Can LLMs Separate Instructions From Data? And What Do We Even Mean By That?cited by 70
  15. 2024Defending Large Language Models against Jailbreak Attacks via Semantic Smoothingcited by 68
  16. 2024Exploring the Adversarial Vulnerabilities of Vision-Language-Action Models in Roboticscited by 68
  17. 2024Speak Out of Turn: Safety Vulnerability of Large Language Models in Multi-turn Dialoguecited by 64
  18. 2024Gradient Cuff: Detecting Jailbreak Attacks on Large Language Models by Exploring Refusal Loss Landscapescited by 63
  19. 2023Jailbreaking GPT-4V via Self-Adversarial Attacks with System Promptscited by 62
  20. 2024Pandora: Jailbreak GPTs by Retrieval Augmented Generation Poisoningcited by 62
  21. 2024SG-Bench: Evaluating LLM Safety Generalization Across Diverse Tasks and Prompt Typescited by 58
  22. 2024Play Guessing Game with LLM: Indirect Jailbreak Attack with Implicit Cluescited by 57
  23. 2024SelfDefend: LLMs Can Defend Themselves against Jailbreaking in a Practical Mannercited by 57
  24. 2024MultiAgent Collaboration Attack: Investigating Adversarial Attacks in Large Language Model Collaborations via Debatecited by 54
  25. 2023From Prompt Injections to SQL Injection Attacks: How Protected is Your LLM-Integrated Web Application?cited by 53
  26. 2023Self-Guard: Empower the LLM to Safeguard Itselfcited by 52
  27. 2024On Evaluating the Durability of Safeguards for Open-Weight LLMscited by 51
  28. 2024Eraser: Jailbreaking Defense in Large Language Models via Unlearning Harmful Knowledgecited by 48
  29. 2024Persistent Pre-Training Poisoning of LLMscited by 46
  30. 2025Can Indirect Prompt Injection Attacks Be Detected and Removed?cited by 43
  31. 2024Signed-Prompt: A New Approach to Prevent Prompt Injection Attacks Against LLM-Integrated Applicationscited by 42
  32. 2024HAICOSYSTEM: An Ecosystem for Sandboxing Safety Risks in Human-AI Interactionscited by 42
  33. 2024Defensive Prompt Patch: A Robust and Interpretable Defense of LLMs against Jailbreak Attackscited by 35
  34. 2024SPML: A DSL for Defending Language Models Against Prompt Attackscited by 33
  35. 2025MCIP: Protecting MCP Safety via Model Contextual Integrity Protocolcited by 33
  36. 2024Human-Imperceptible Retrieval Poisoning Attacks in LLM-Powered Applicationscited by 32
  37. 2025Exploring the Limits of Vision-Language-Action Manipulations in Cross-task Generalizationcited by 31
  38. 2024PARDEN, Can You Repeat That? Defending against Jailbreaks via Repetitioncited by 29

50 most cited of 67 All 67 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026AuthorityLens: Rethinking LLM-Based Agent Systems Through the Lens of Authoritycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026SoK: When Safe Agents Fail Together: The Security of Multi Agent LLM Systems
  2. 2025More Vulnerable than You Think: On the Stability of Tool-Integrated LLM Agents
  3. 2026Safe Multi-Agent Behavior Must Be Maintained, Not Merely Asserted: Constraint Drift in LLM-Based Multi-Agent Systems
  4. 2024IsolateGPT: An Execution Isolation Architecture for LLM-Based Agentic Systems
  5. 2025Beyond Single-Agent Safety: A Taxonomy of Risks in LLM-to-LLM Interactions
  6. 2025DRIFT: Dynamic Rule-Based Defense with Injection Isolation for Securing LLM Agents
  7. 2026Position: A Three-Layer Probabilistic Assume-Guarantee Architecture Is Structurally Required for Safe LLM Agent Deployment
  8. 2026Toward Safe LLM Agents: A Survey of Specification, Verification, and Enforcement