DRIFT: Dynamic Rule-Based Defense with Injection Isolation for Securing LLM Agents
On the map: LLM agents and tool use → Agent threats, guardrails and governance
also in 1 more
Abstract (arXiv)
Large Language Models (LLMs) are increasingly central to agentic systems due to their strong reasoning and planning capabilities. By interacting with external environments through predefined tools, these agents can carry out complex user tasks. Nonetheless, this interaction also introduces the risk of prompt injection attacks, where malicious inputs from external sources can mislead the agent's behavior, potentially resulting in economic loss, privacy leakage, or system compromise. System-level defenses have recently shown promise by enforcing static or predefined policies, but they still face two key challenges: the ability to dynamically update security rules and the need for memory stream isolation. To address these challenges, we propose Dynamic Rule-based Isolation Framework for Trustworthy agentic systems (DRIFT), which enforces the dynamic security policy and injection isolation for securing LLM agents against prompt injection attacks. A Secure Planner first constructs a minimal function trajectory and a JSON-schema-style parameter checklist for each function node based on the user query. A Dynamic Validator then monitors deviations from the original plan, assessing whether changes comply with privilege limitations and the user's intent. Finally, an Injection Isolator detects and masks any instructions that may conflict with the user query from the memory stream to mitigate long-term risks. We empirically validate the effectiveness of DRIFT on the AgentDojo, ASB, and AgentDyn benchmark, demonstrating its strong security performance while maintaining high utility across diverse models, showcasing both its robustness and adaptability. The project website is available at https://safo-lab.github.io/DRIFT.
Key facts. Cited by 66 papers in the map (data through 14 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 36 of its 49 references that are in the map. First posted on arXiv 13 Jun 2025.
Builds on
36 of 49 references.
Closest ancestors · 7
Same direction, newest first
- 2025Defeating Prompt Injections by Design24 Mar
- 2025RTBAS: Defending LLM Agents Against Prompt Injection and Privacy Leakage13 Feb
- 2024Attacking Vision-Language Computer Agents via Pop-ups4 Nov
- 2024Agent Security Bench (ASB): Formalizing and Benchmarking Attacks and Defenses in LLM-based Agents3 Oct
- 2024EIA: Environmental Injection Attack on Generalist Web Agents for Privacy Leakage17 Sep
- 2024IsolateGPT: An Execution Isolation Architecture for LLM-Based Agentic Systems8 Mar
- 2024InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents5 Mar
Other references · 29
- 2014Adam: A Method for Stochastic Optimizationcited by 37,240
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
- 2024Qwen2.5 Technical Reportcited by 4,942
- 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 2,006
- 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,947
- 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,737
- 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,443
- 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,420
- 2023Mind2Web: Towards a Generalist Agent for the Webcited by 1,242
- 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,207
- 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,195
17 more
- 2022Ignore Previous Prompt: Attack Techniques For Language Modelscited by 832
- 2023Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Modelscited by 574
- 2024StruQ: Defending Against Prompt Injection with Structured Queriescited by 375
- 2023A Real-World WebAgent with Planning, Long Context Understanding, and Program Synthesiscited by 353
- 2024Defending Against Indirect Prompt Injection Attacks With Spotlightingcited by 267
- 2023MetaTool Benchmark for Large Language Models: Deciding Whether to Use Tools and Which to Usecited by 217
- 2024SecAlign: Defending Against Prompt Injection with Preference Optimizationcited by 210
- 2023TaskMatrix.AI: Completing Tasks by Connecting Foundation Models with Millions of APIscited by 184
- 2023QA-LoRA: Quantization-Aware Low-Rank Adaptation of Large Language Modelscited by 155
- 2024Hello Again! LLM-powered Personalized Agent for Long-term Dialoguecited by 123
- 2024Automatic and Universal Prompt Injection Attacks against Large Language Modelscited by 119
- 2024System-Level Defense against Indirect Prompt Injection Attacks: An Information Flow Control Perspectivecited by 96
- 2023On Generative Agents in Recommendationcited by 90
- 2024Neural Exec: Learning (and Learning from) Execution Triggers for Prompt Injection Attackscited by 85
- 2024Signed-Prompt: A New Approach to Prevent Prompt Injection Attacks Against LLM-Integrated Applicationscited by 54
- 2024InjecGuard: Benchmarking and Mitigating Over-defense in Prompt Injection Guardrail Modelscited by 42
- 2025CAMEL: Continuous Action Masking Enabled by Large Language Models for Reinforcement Learningcited by 2
Built on by
Citing papers per year: 2025: 8; 2026: 58.
66 papers.
Sort:
By area:
- 2025The Attacker Moves Second: Stronger Adaptive Attacks Bypass Defenses Against Llm Jailbreaks and Prompt Injectionscited by 129
- 2025A Survey of LLM-Driven AI Agent Communication: Protocols, Security Risks, and Defense Countermeasurescited by 57
- 2025AgentArmor: Enforcing Program Analysis on Agent Runtime Trace to Defend Against Prompt Injectioncited by 46
- 2026Memory Poisoning Attack and Defense on Memory Based LLM-Agentscited by 44
- 2026The Attack and Defense Landscape of Agentic AI: A Comprehensive Surveycited by 28
- 2026VIGIL: Defending LLM Agents Against Tool Stream Injection via Verify-Before-Commitcited by 17
- 2026PISmith: Reinforcement Learning-based Red Teaming for Prompt Injection Defensescited by 16
- 2026A Framework for Formalizing LLM Agent Securitycited by 16
- 2026Agent-Sentry: Bounding LLM Agents via Execution Provenancecited by 15
- 2026Aligning Provenance with Authorization: A Dual-Graph Defense for LLM Agentscited by 15
- 2026CausalArmor: Efficient Indirect Prompt Injection Guardrails via Causal Attributioncited by 14
- 2026ARGUS: Defending LLM Agents Against Context-Aware Prompt Injectioncited by 14
38 more
- 2025Systems Security Foundations for Agentic Computingcited by 13
- 2025Breaking and Fixing Defenses Against Control-Flow Hijacking in Multi-Agent Systemscited by 12
- 2026AgentGuardian: Learning Access Control Policies to Govern AI Agent Behaviorcited by 11
- 2026The Granularity Mismatch in Agent Security: Argument-Level Provenance Solves Enforcement and Isolates the LLM Reasoning Bottleneckcited by 11
- 2026AgentSys: Secure and Dynamic LLM Agents Through Explicit Hierarchical Memory Managementcited by 10
- 2025Taxonomy, Evaluation and Exploitation of IPI-Centric LLM Agent Defense Frameworkscited by 9
- 2026Architecting Secure AI Agents: Perspectives on System-Level Defenses Against Indirect Prompt Injection Attackscited by 9
- 2026Reframing LLM Agent Security as an Agent-Human Interaction Problemcited by 9
- 2026Security Considerations for Artificial Intelligence Agentscited by 8
- 2026Safeguarding LLM Agents against Long-Horizon Threats via Shadow Memorycited by 7
- 2026Toward Secure LLM Agents: Threat Surfaces, Attacks, Defenses, and Evaluationcited by 7
- 2026AgentWatcher: A Rule-based Prompt Injection Monitorcited by 6
- 2026SMCP: Secure Model Context Protocolcited by 5
- 2026SkillScope: Toward Fine-Grained Least-Privilege Enforcement for Agent Skillscited by 5
- 2026AutoDojo: Adaptive Black-Box Attacks Reveal the Limits of IPI Defenses and Task-Specification Effects in LLM Agentscited by 5
- 2026How Agents Ask for Permission: User Permissions for AI Agents, from Interfaces to Enforcementcited by 4
- 2026Cross-Layer Semantic Flow Reconstruction for Attack Detection in Agentic Systemscited by 3
- 2026Securing LLM Agents Need Intent-to-Execution Integritycited by 3
- 2026Think Twice Before You Act: Protecting LLM Agents Against Tool Description Poisoning via Isolated Planningcited by 3
- 2025AgentCrypt: Advancing Privacy and (Secure) Computation in AI Agent Collaborationcited by 2
- 2026AgentShield: Deception-based Compromise Detection for Tool-using LLM Agentscited by 2
- 2026SecureClaw: Clawing Back Control of LLM Agentscited by 2
- 2026Efficient and Sound Probabilistic Verification for AI Agentscited by 2
- 2026Behavioral Skill Reconstruction: Reconstructing Hidden Functionality from LLM Agent Skillscited by 2
- 2026ROPE: Routed Origin Policy Enforcement against Indirect Prompt Injectioncited by 2
- 2026EvoSafeHarness: Evolving Model- and Domain-Specific Harnesses for Securing Agentscited by 2
- 2026CoER: Defending against Adaptive Indirect Prompt Injection via Adversarial Co-Evolution and Refinementcited by 2
- 2026AgentAntibody: An Adaptive Immune System for Defending LLM Agents against Prompt Injectioncited by 1
- 2026Agent Against Agent: An Agentic System for Automatic Prompt Injection Red Teamingcited by 1
- 2026Robust Context-Aware Detection of Malicious Instructions in Textcited by 1
- 2026ActGov: Governing LLM Agent Actions via Policy-Constrained Validationcited by 1
- 2025SIEVE: Selective Integrity Verification and Escalation for Defending LLM Agents against Indirect Prompt Injectioncited by 0
- 2026Blue Teaming Function-Calling Agentscited by 0
- 2026Relevance as a Vulnerability: How Web Retrieval Degrades Safety Alignment in LLM Agentscited by 0
- 2026Where Instruction Hierarchy Breaks: Diagnosing and Repairing Failures in Reasoning Language Modelscited by 0
- 2026MOSAIC: Knowledge-Guided CLI Command Composition Attack in LLM Coding Agentscited by 0
- 2026DualView: Preventing Indirect Prompt Injection in Personal AI Agentscited by 0
- 2026Cross-Agent Campaign Attribution: Linking Asynchronous Attacks Across LLM Agentscited by 0
50 most cited of 66 All 66 (JSON)
- 2026Holding Agents Accountable with Verifiable Agent Insurance13 Oct 2026
- 2026Secure-CUA: Controlling Untrusted Influence in Computer-Use Agents7 Oct 2026
- 2026APEX: Active Protection at Execution Boundaries for LLM Agents3 Oct 2026
- 2026Self-Reflection Fine-Tuning: Enhancing Agent Security against Prompt Injection Attacks from Failure Experience3 Oct 2026
- 2026Pincer: Resource Authorization for Agents using a Digital Twin1 Oct 2026
- 2026PACE: Provenance-Aware Capability Enforcement for Tool-Using LLM Agents1 Oct 2026
- 2026ActionGuard: Tool Call Authorization under Poisoned Skills30 Sep 2026
- 2026CounterSteer: Suppressing Indirect Prompt Injection with Activation Steering29 Sep 2026
- 2026Beyond Approved Actions: Runtime Validation of Persistent Outcomes in Agent Workflows25 Sep 2026
- 2026ActGov: Governing LLM Agent Actions via Policy-Constrained Validation21 Sep 2026
- 2026Environment Steering: Using Data Flow Control to Improve Agent Utility and Safety19 Sep 2026
- 2026ActGuard: Pre-execution Action Auditing against Indirect Prompt Injection in LLM Agents14 Sep 2026
38 more
- 2026CoER: Defending against Adaptive Indirect Prompt Injection via Adversarial Co-Evolution and Refinement7 Sep 2026
- 2026Skynet: Workflow-Level Anomaly Detection for Agentic AI via Semantic and Structural Modeling6 Sep 2026
- 2026EvoSafeHarness: Evolving Model- and Domain-Specific Harnesses for Securing Agents5 Sep 2026
- 2026ROPE: Routed Origin Policy Enforcement against Indirect Prompt Injection27 Aug 2026
- 2026A Policy Algebra for Trust-Preserving Agentic AI Execution17 Aug 2026
- 2026Beyond Handcrafted Security: Towards Self-Evolving Defense for LLM Agents13 Aug 2026
- 2026PIPES: Securing Agent Perception with Provenance and Priors13 Aug 2026
- 2026Robust Context-Aware Detection of Malicious Instructions in Text5 Aug 2026
- 2026Agent Against Agent: An Agentic System for Automatic Prompt Injection Red Teaming5 Aug 2026
- 2026Behavioral Skill Reconstruction: Reconstructing Hidden Functionality from LLM Agent Skills4 Aug 2026
- 2026AgentAntibody: An Adaptive Immune System for Defending LLM Agents against Prompt Injection4 Aug 2026
- 2026MAPLE-Guard: Memory-Aware Link Enforcement Against Memory-Link Poisoning in Multi-Agent Systems1 Aug 2026
- 2026Cross-Agent Campaign Attribution: Linking Asynchronous Attacks Across LLM Agents21 Jul 2026
- 2026How Agents Ask for Permission: User Permissions for AI Agents, from Interfaces to Enforcement15 Jul 2026
- 2026DualView: Preventing Indirect Prompt Injection in Personal AI Agents4 Jul 2026
- 2026MOSAIC: Knowledge-Guided CLI Command Composition Attack in LLM Coding Agents3 Jul 2026
- 2026Think Twice Before You Act: Protecting LLM Agents Against Tool Description Poisoning via Isolated Planning18 Jun 2026
- 2026Efficient and Sound Probabilistic Verification for AI Agents18 Jun 2026
- 2026AutoDojo: Adaptive Black-Box Attacks Reveal the Limits of IPI Defenses and Task-Specification Effects in LLM Agents13 Jun 2026
- 2026Toward Secure LLM Agents: Threat Surfaces, Attacks, Defenses, and Evaluation9 Jun 2026
- 2026SecureClaw: Clawing Back Control of LLM Agents8 Jun 2026
- 2026Where Instruction Hierarchy Breaks: Diagnosing and Repairing Failures in Reasoning Language Models5 Jun 2026
- 2026Relevance as a Vulnerability: How Web Retrieval Degrades Safety Alignment in LLM Agents28 May 2026
- 2026Aligning Provenance with Authorization: A Dual-Graph Defense for LLM Agents26 May 2026
- 2026Reframing LLM Agent Security as an Agent-Human Interaction Problem23 May 2026
- 2026Securing LLM Agents Need Intent-to-Execution Integrity16 May 2026
- 2026The Granularity Mismatch in Agent Security: Argument-Level Provenance Solves Enforcement and Isolates the LLM Reasoning Bottleneck11 May 2026
- 2026AgentShield: Deception-based Compromise Detection for Tool-using LLM Agents10 May 2026
- 2026SkillScope: Toward Fine-Grained Least-Privilege Enforcement for Agent Skills7 May 2026
- 2026ARGUS: Defending LLM Agents Against Context-Aware Prompt Injection5 May 2026
- 2026Safeguarding LLM Agents against Long-Horizon Threats via Shadow Memory4 May 2026
- 2026AgentWatcher: A Rule-based Prompt Injection Monitor1 Apr 2026
- 2026Architecting Secure AI Agents: Perspectives on System-Level Defenses Against Indirect Prompt Injection Attacks31 Mar 2026
- 2026Agent-Sentry: Bounding LLM Agents via Execution Provenance24 Mar 2026
- 2026A Framework for Formalizing LLM Agent Security19 Mar 2026
- 2026PISmith: Reinforcement Learning-based Red Teaming for Prompt Injection Defenses13 Mar 2026
- 2026Security Considerations for Artificial Intelligence Agents12 Mar 2026
- 2026The Attack and Defense Landscape of Agentic AI: A Comprehensive Survey11 Mar 2026
50 newest of 66 All 66 (JSON)
- 2025A Survey of LLM-Driven AI Agent Communication: Protocols, Security Risks, and Defense Countermeasurescited by 57
- 2025AgentArmor: Enforcing Program Analysis on Agent Runtime Trace to Defend Against Prompt Injectioncited by 46
- 2026Memory Poisoning Attack and Defense on Memory Based LLM-Agentscited by 44
- 2026The Attack and Defense Landscape of Agentic AI: A Comprehensive Surveycited by 28
- 2026VIGIL: Defending LLM Agents Against Tool Stream Injection via Verify-Before-Commitcited by 17
- 2026A Framework for Formalizing LLM Agent Securitycited by 16
- 2026Agent-Sentry: Bounding LLM Agents via Execution Provenancecited by 15
- 2026Aligning Provenance with Authorization: A Dual-Graph Defense for LLM Agentscited by 15
- 2026CausalArmor: Efficient Indirect Prompt Injection Guardrails via Causal Attributioncited by 14
- 2026ARGUS: Defending LLM Agents Against Context-Aware Prompt Injectioncited by 14
- 2025Systems Security Foundations for Agentic Computingcited by 13
- 2025Breaking and Fixing Defenses Against Control-Flow Hijacking in Multi-Agent Systemscited by 12
12 most cited of 55 All 55 (JSON)
- 2026Secure-CUA: Controlling Untrusted Influence in Computer-Use Agents7 Oct 2026
- 2026APEX: Active Protection at Execution Boundaries for LLM Agents3 Oct 2026
- 2026Self-Reflection Fine-Tuning: Enhancing Agent Security against Prompt Injection Attacks from Failure Experience3 Oct 2026
- 2026Pincer: Resource Authorization for Agents using a Digital Twin1 Oct 2026
- 2026PACE: Provenance-Aware Capability Enforcement for Tool-Using LLM Agents1 Oct 2026
- 2026ActionGuard: Tool Call Authorization under Poisoned Skills30 Sep 2026
- 2026CounterSteer: Suppressing Indirect Prompt Injection with Activation Steering29 Sep 2026
- 2026Beyond Approved Actions: Runtime Validation of Persistent Outcomes in Agent Workflows25 Sep 2026
- 2026ActGov: Governing LLM Agent Actions via Policy-Constrained Validation21 Sep 2026
- 2026Environment Steering: Using Data Flow Control to Improve Agent Utility and Safety19 Sep 2026
- 2026ActGuard: Pre-execution Action Auditing against Indirect Prompt Injection in LLM Agents14 Sep 2026
- 2026CoER: Defending against Adaptive Indirect Prompt Injection via Adversarial Co-Evolution and Refinement7 Sep 2026
12 newest of 55 All 55 (JSON)
- 2025The Attacker Moves Second: Stronger Adaptive Attacks Bypass Defenses Against Llm Jailbreaks and Prompt Injectionscited by 129
- 2026PISmith: Reinforcement Learning-based Red Teaming for Prompt Injection Defensescited by 16
- 2026AgentWatcher: A Rule-based Prompt Injection Monitorcited by 6
- 2026Robust Context-Aware Detection of Malicious Instructions in Textcited by 1
- 2026Robust Context-Aware Detection of Malicious Instructions in Text5 Aug 2026
- 2026AgentWatcher: A Rule-based Prompt Injection Monitor1 Apr 2026
- 2026PISmith: Reinforcement Learning-based Red Teaming for Prompt Injection Defenses13 Mar 2026
- 2025The Attacker Moves Second: Stronger Adaptive Attacks Bypass Defenses Against Llm Jailbreaks and Prompt Injections10 Oct 2025
- 2026AgentGuardian: Learning Access Control Policies to Govern AI Agent Behaviorcited by 11
- 2026Security Considerations for Artificial Intelligence Agentscited by 8
- 2026Efficient and Sound Probabilistic Verification for AI Agentscited by 2
- 2026PIPES: Securing Agent Perception with Provenance and Priorscited by 0
- 2026Holding Agents Accountable with Verifiable Agent Insurancecited by 0
- 2026Holding Agents Accountable with Verifiable Agent Insurance13 Oct 2026
- 2026PIPES: Securing Agent Perception with Provenance and Priors13 Aug 2026
- 2026Efficient and Sound Probabilistic Verification for AI Agents18 Jun 2026
- 2026Security Considerations for Artificial Intelligence Agents12 Mar 2026
- 2026AgentGuardian: Learning Access Control Policies to Govern AI Agent Behavior15 Jan 2026
Similar, not cited
Close in wording, no citation either way
- 2026PlanGuard: Defending Agents against Indirect Prompt Injection via Planning-based Consistency Verification
- 2026The Landscape of Prompt Injection Threats in LLM Agents: From Taxonomy to Analysis
- 2026AgentVisor: Defending LLM Agents Against Prompt Injection via Semantic Virtualization
- 2025Prompt Flow Integrity to Prevent Privilege Escalation in LLM Agents
- 2025The Dark Side of LLMs: Agent-based Attack Vectors for System-level Compromise
- 2025Design Patterns for Securing LLM Agents against Prompt Injections
- 2025Preventing Prompt Injection with Type-Directed Privilege Separation
- 2026SafeHarness: Lifecycle-Integrated Security Architecture for LLM-based Agent Deployment