mlsanityData to 14 Oct 2026

DRIFT: Dynamic Rule-Based Defense with Injection Isolation for Securing LLM Agents

arXiv 2506.12104 ↗First posted 13 Jun 2025cs.CR cs.AIcited by 66

On the map: LLM agents and tool use → Agent threats, guardrails and governance

also in 1 more
Abstract (arXiv)

Large Language Models (LLMs) are increasingly central to agentic systems due to their strong reasoning and planning capabilities. By interacting with external environments through predefined tools, these agents can carry out complex user tasks. Nonetheless, this interaction also introduces the risk of prompt injection attacks, where malicious inputs from external sources can mislead the agent's behavior, potentially resulting in economic loss, privacy leakage, or system compromise. System-level defenses have recently shown promise by enforcing static or predefined policies, but they still face two key challenges: the ability to dynamically update security rules and the need for memory stream isolation. To address these challenges, we propose Dynamic Rule-based Isolation Framework for Trustworthy agentic systems (DRIFT), which enforces the dynamic security policy and injection isolation for securing LLM agents against prompt injection attacks. A Secure Planner first constructs a minimal function trajectory and a JSON-schema-style parameter checklist for each function node based on the user query. A Dynamic Validator then monitors deviations from the original plan, assessing whether changes comply with privilege limitations and the user's intent. Finally, an Injection Isolator detects and masks any instructions that may conflict with the user query from the memory stream to mitigate long-term risks. We empirically validate the effectiveness of DRIFT on the AgentDojo, ASB, and AgentDyn benchmark, demonstrating its strong security performance while maintaining high utility across diverse models, showcasing both its robustness and adaptability. The project website is available at https://safo-lab.github.io/DRIFT.

Key facts. Cited by 66 papers in the map (data through 14 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 36 of its 49 references that are in the map. First posted on arXiv 13 Jun 2025.

Builds on

36 of 49 references.

Closest ancestors · 7

Same direction, newest first

  1. 2025Defeating Prompt Injections by Design24 Mar
  2. 2025RTBAS: Defending LLM Agents Against Prompt Injection and Privacy Leakage13 Feb
  3. 2024Attacking Vision-Language Computer Agents via Pop-ups4 Nov
  4. 2024Agent Security Bench (ASB): Formalizing and Benchmarking Attacks and Defenses in LLM-based Agents3 Oct
  5. 2024EIA: Environmental Injection Attack on Generalist Web Agents for Privacy Leakage17 Sep
  6. 2024IsolateGPT: An Execution Isolation Architecture for LLM-Based Agentic Systems8 Mar
  7. 2024InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents5 Mar

Other references · 29

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  4. 2024Qwen2.5 Technical Reportcited by 4,942
  5. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 2,006
  6. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,947
  7. 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,737
  8. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,443
  9. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,420
  10. 2023Mind2Web: Towards a Generalist Agent for the Webcited by 1,242
  11. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,207
  12. 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,195
17 more
  1. 2022Ignore Previous Prompt: Attack Techniques For Language Modelscited by 832
  2. 2023Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Modelscited by 574
  3. 2024StruQ: Defending Against Prompt Injection with Structured Queriescited by 375
  4. 2023A Real-World WebAgent with Planning, Long Context Understanding, and Program Synthesiscited by 353
  5. 2024Defending Against Indirect Prompt Injection Attacks With Spotlightingcited by 267
  6. 2023MetaTool Benchmark for Large Language Models: Deciding Whether to Use Tools and Which to Usecited by 217
  7. 2024SecAlign: Defending Against Prompt Injection with Preference Optimizationcited by 210
  8. 2023TaskMatrix.AI: Completing Tasks by Connecting Foundation Models with Millions of APIscited by 184
  9. 2023QA-LoRA: Quantization-Aware Low-Rank Adaptation of Large Language Modelscited by 155
  10. 2024Hello Again! LLM-powered Personalized Agent for Long-term Dialoguecited by 123
  11. 2024Automatic and Universal Prompt Injection Attacks against Large Language Modelscited by 119
  12. 2024System-Level Defense against Indirect Prompt Injection Attacks: An Information Flow Control Perspectivecited by 96
  13. 2023On Generative Agents in Recommendationcited by 90
  14. 2024Neural Exec: Learning (and Learning from) Execution Triggers for Prompt Injection Attackscited by 85
  15. 2024Signed-Prompt: A New Approach to Prevent Prompt Injection Attacks Against LLM-Integrated Applicationscited by 54
  16. 2024InjecGuard: Benchmarking and Mitigating Over-defense in Prompt Injection Guardrail Modelscited by 42
  17. 2025CAMEL: Continuous Action Masking Enabled by Large Language Models for Reinforcement Learningcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 8; 2026: 58.

66 papers.

Sort:

By area:

  1. 2025The Attacker Moves Second: Stronger Adaptive Attacks Bypass Defenses Against Llm Jailbreaks and Prompt Injectionscited by 129
  2. 2025A Survey of LLM-Driven AI Agent Communication: Protocols, Security Risks, and Defense Countermeasurescited by 57
  3. 2025AgentArmor: Enforcing Program Analysis on Agent Runtime Trace to Defend Against Prompt Injectioncited by 46
  4. 2026Memory Poisoning Attack and Defense on Memory Based LLM-Agentscited by 44
  5. 2026The Attack and Defense Landscape of Agentic AI: A Comprehensive Surveycited by 28
  6. 2026VIGIL: Defending LLM Agents Against Tool Stream Injection via Verify-Before-Commitcited by 17
  7. 2026PISmith: Reinforcement Learning-based Red Teaming for Prompt Injection Defensescited by 16
  8. 2026A Framework for Formalizing LLM Agent Securitycited by 16
  9. 2026Agent-Sentry: Bounding LLM Agents via Execution Provenancecited by 15
  10. 2026Aligning Provenance with Authorization: A Dual-Graph Defense for LLM Agentscited by 15
  11. 2026CausalArmor: Efficient Indirect Prompt Injection Guardrails via Causal Attributioncited by 14
  12. 2026ARGUS: Defending LLM Agents Against Context-Aware Prompt Injectioncited by 14
38 more
  1. 2025Systems Security Foundations for Agentic Computingcited by 13
  2. 2025Breaking and Fixing Defenses Against Control-Flow Hijacking in Multi-Agent Systemscited by 12
  3. 2026AgentGuardian: Learning Access Control Policies to Govern AI Agent Behaviorcited by 11
  4. 2026The Granularity Mismatch in Agent Security: Argument-Level Provenance Solves Enforcement and Isolates the LLM Reasoning Bottleneckcited by 11
  5. 2026AgentSys: Secure and Dynamic LLM Agents Through Explicit Hierarchical Memory Managementcited by 10
  6. 2025Taxonomy, Evaluation and Exploitation of IPI-Centric LLM Agent Defense Frameworkscited by 9
  7. 2026Architecting Secure AI Agents: Perspectives on System-Level Defenses Against Indirect Prompt Injection Attackscited by 9
  8. 2026Reframing LLM Agent Security as an Agent-Human Interaction Problemcited by 9
  9. 2026Security Considerations for Artificial Intelligence Agentscited by 8
  10. 2026Safeguarding LLM Agents against Long-Horizon Threats via Shadow Memorycited by 7
  11. 2026Toward Secure LLM Agents: Threat Surfaces, Attacks, Defenses, and Evaluationcited by 7
  12. 2026AgentWatcher: A Rule-based Prompt Injection Monitorcited by 6
  13. 2026SMCP: Secure Model Context Protocolcited by 5
  14. 2026SkillScope: Toward Fine-Grained Least-Privilege Enforcement for Agent Skillscited by 5
  15. 2026AutoDojo: Adaptive Black-Box Attacks Reveal the Limits of IPI Defenses and Task-Specification Effects in LLM Agentscited by 5
  16. 2026How Agents Ask for Permission: User Permissions for AI Agents, from Interfaces to Enforcementcited by 4
  17. 2026Cross-Layer Semantic Flow Reconstruction for Attack Detection in Agentic Systemscited by 3
  18. 2026Securing LLM Agents Need Intent-to-Execution Integritycited by 3
  19. 2026Think Twice Before You Act: Protecting LLM Agents Against Tool Description Poisoning via Isolated Planningcited by 3
  20. 2025AgentCrypt: Advancing Privacy and (Secure) Computation in AI Agent Collaborationcited by 2
  21. 2026AgentShield: Deception-based Compromise Detection for Tool-using LLM Agentscited by 2
  22. 2026SecureClaw: Clawing Back Control of LLM Agentscited by 2
  23. 2026Efficient and Sound Probabilistic Verification for AI Agentscited by 2
  24. 2026Behavioral Skill Reconstruction: Reconstructing Hidden Functionality from LLM Agent Skillscited by 2
  25. 2026ROPE: Routed Origin Policy Enforcement against Indirect Prompt Injectioncited by 2
  26. 2026EvoSafeHarness: Evolving Model- and Domain-Specific Harnesses for Securing Agentscited by 2
  27. 2026CoER: Defending against Adaptive Indirect Prompt Injection via Adversarial Co-Evolution and Refinementcited by 2
  28. 2026AgentAntibody: An Adaptive Immune System for Defending LLM Agents against Prompt Injectioncited by 1
  29. 2026Agent Against Agent: An Agentic System for Automatic Prompt Injection Red Teamingcited by 1
  30. 2026Robust Context-Aware Detection of Malicious Instructions in Textcited by 1
  31. 2026ActGov: Governing LLM Agent Actions via Policy-Constrained Validationcited by 1
  32. 2025SIEVE: Selective Integrity Verification and Escalation for Defending LLM Agents against Indirect Prompt Injectioncited by 0
  33. 2026Blue Teaming Function-Calling Agentscited by 0
  34. 2026Relevance as a Vulnerability: How Web Retrieval Degrades Safety Alignment in LLM Agentscited by 0
  35. 2026Where Instruction Hierarchy Breaks: Diagnosing and Repairing Failures in Reasoning Language Modelscited by 0
  36. 2026MOSAIC: Knowledge-Guided CLI Command Composition Attack in LLM Coding Agentscited by 0
  37. 2026DualView: Preventing Indirect Prompt Injection in Personal AI Agentscited by 0
  38. 2026Cross-Agent Campaign Attribution: Linking Asynchronous Attacks Across LLM Agentscited by 0

50 most cited of 66 All 66 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026PlanGuard: Defending Agents against Indirect Prompt Injection via Planning-based Consistency Verification
  2. 2026The Landscape of Prompt Injection Threats in LLM Agents: From Taxonomy to Analysis
  3. 2026AgentVisor: Defending LLM Agents Against Prompt Injection via Semantic Virtualization
  4. 2025Prompt Flow Integrity to Prevent Privilege Escalation in LLM Agents
  5. 2025The Dark Side of LLMs: Agent-based Attack Vectors for System-level Compromise
  6. 2025Design Patterns for Securing LLM Agents against Prompt Injections
  7. 2025Preventing Prompt Injection with Type-Directed Privilege Separation
  8. 2026SafeHarness: Lifecycle-Integrated Security Architecture for LLM-based Agent Deployment