mlsanityData to 14 Oct 2026

RTBAS: Defending LLM Agents Against Prompt Injection and Privacy Leakage

arXiv 2502.08966 ↗First posted 13 Feb 2025cs.CR cs.AIcited by 69

On the map: LLM agents and tool use → Agent threats, guardrails and governance

also in 1 more
Abstract (arXiv)

Tool-Based Agent Systems (TBAS) allow Language Models (LMs) to use external tools for tasks beyond their standalone capabilities, such as searching websites, booking flights, or making financial transactions. However, these tools greatly increase the risks of prompt injection attacks, where malicious content hijacks the LM agent to leak confidential data or trigger harmful actions. Existing defenses (OpenAI GPTs) require user confirmation before every tool call, placing onerous burdens on users. We introduce Robust TBAS (RTBAS), which automatically detects and executes tool calls that preserve integrity and confidentiality, requiring user confirmation only when these safeguards cannot be ensured. RTBAS adapts Information Flow Control to the unique challenges presented by TBAS. We present two novel dependency screeners, using LM-as-a-judge and attention-based saliency, to overcome these challenges. Experimental results on the AgentDojo Prompt Injection benchmark show RTBAS prevents all targeted attacks with only a 2% loss of task utility when under attack, and further tests confirm its ability to obtain near-oracle performance on detecting both subtle and direct privacy leaks.

Key facts. Cited by 69 papers in the map (data through 14 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 20 of its 46 references that are in the map. First posted on arXiv 13 Feb 2025.

Builds on

20 of 46 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents19 Jun
  2. 2024InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents5 Mar

Other references · 18

  1. 2017Attention Is All You Needcited by 60,361
  2. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  3. 2020Extracting Training Data from Large Language Modelscited by 2,189
  4. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,917
  5. 2019Attention is not Explanationcited by 929
  6. 2023H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Modelscited by 832
  7. 2019Are Sixteen Heads Really Better than One?cited by 816
  8. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Modelscited by 785
  9. 2019Attention is not not Explanationcited by 636
  10. 2023Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Modelscited by 560
  11. 2023Formalizing and Benchmarking Prompt Injection Attacks and Defensescited by 364
  12. 2023Label Words are Anchors: An Information Flow Perspective for Understanding In-Context Learningcited by 236
6 more
  1. 2023ProPILE: Probing Privacy Leakage in Large Language Modelscited by 129
  2. 2024Attention Tracker: Detecting Prompt Injection Attacks in LLMscited by 95
  3. 2024The Benefits of a Concise Chain of Thought on Problem-Solving in Large Language Modelscited by 63
  4. 2024PAPILLON: Privacy Preservation from Internet-based and Local Language Model Ensemblescited by 57
  5. 2024CLEAR: Towards Contextual LLM-Empowered Privacy Policy Analysis and Risk Generation for Large Language Model Applicationscited by 33
  6. 2024Fine-tuned Large Language Models (LLMs): Improved Prompt Injection Attacks Detectioncited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 17; 2026: 52.

69 papers.

Sort:

By area:

  1. 2025Defeating Prompt Injections by Designcited by 297
  2. 2025Large Language Model Agent: A Survey on Methodology, Applications and Challengescited by 205
  3. 2025Securing AI Agents with Information-Flow Controlcited by 121
  4. 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 116
  5. 2025A Survey of WebAgents: Towards Next-Generation AI Agents for Web Automation with Large Foundation Modelscited by 87
  6. 2025DRIFT: Dynamic Rule-Based Defense with Injection Isolation for Securing LLM Agentscited by 66
  7. 2025AgentArmor: Enforcing Program Analysis on Agent Runtime Trace to Defend Against Prompt Injectioncited by 46
  8. 2026Towards Verifiably Safe Tool Use for LLM Agentscited by 36
  9. 2026The Landscape of Prompt Injection Threats in LLM Agents: From Taxonomy to Analysiscited by 30
  10. 2025LLM Agents Should Employ Security Principlescited by 29
  11. 2026The Attack and Defense Landscape of Agentic AI: A Comprehensive Surveycited by 28
  12. 2025Simple Prompt Injection Attacks Can Leak Personal Data Observed by LLM Agents During Task Executioncited by 26
38 more
  1. 2025A Survey on the Safety and Security Threats of Computer-Using Agents: JARVIS or Ultron?cited by 23
  2. 2025TopicAttack: An Indirect Prompt Injection Attack via Topic Transitioncited by 17
  3. 2026CaMeLs Can Use Computers Too: System-level Security for Computer Use Agentscited by 17
  4. 2025Robustness via Referencing: Defending against Prompt Injection Attacks by Referencing the Executed Instructioncited by 16
  5. 2026Ghost in the Agent: Redefining Information Flow Tracking for LLM Agentscited by 15
  6. 2026Aligning Provenance with Authorization: A Dual-Graph Defense for LLM Agentscited by 15
  7. 2025Systems Security Foundations for Agentic Computingcited by 13
  8. 2026AIRGuard: Guarding Agent Actions with Runtime Authority Controlcited by 13
  9. 2025Position: The Real Barrier to LLM Agent Usability is Agentic ROIcited by 12
  10. 2025Secure and Efficient Access Control for Computer-Use Agents via Context Spacecited by 11
  11. 2026AgentGuardian: Learning Access Control Policies to Govern AI Agent Behaviorcited by 11
  12. 2026AgentSys: Secure and Dynamic LLM Agents Through Explicit Hierarchical Memory Managementcited by 10
  13. 2026Optimizing Agent Planning for Security and Autonomycited by 10
  14. 2026Cloak and Detonate: Scanner Evasion and Dynamic Detection of Agent Skill Malwarecited by 10
  15. 2026Defense Against Indirect Prompt Injection via Tool Result Parsingcited by 9
  16. 2026The Promptware Kill Chain: How Prompt Injections Gradually Evolved Into a Multistep Malware Delivery Mechanismcited by 9
  17. 2026Reframing LLM Agent Security as an Agent-Human Interaction Problemcited by 9
  18. 2026Semia: Auditing Agent Skills via Constraint-Guided Representation Synthesiscited by 7
  19. 2026Assessing Automated Prompt Injection Attacks in Agentic Environmentscited by 7
  20. 2026Toward Secure LLM Agents: Threat Surfaces, Attacks, Defenses, and Evaluationcited by 7
  21. 2026ToolPrivacyBench: Benchmarking Purpose-Bound Privacy in Tool-Using LLM Agentscited by 7
  22. 2026An AI Agent Execution Environment to Safeguard User Datacited by 4
  23. 2025Enterprise AI Must Enforce Participant-Aware Access Controlcited by 3
  24. 2026Clawed and Dangerous: Can We Trust Open Agentic Systems?cited by 3
  25. 2026STARS: Skill-Triggered Audit for Request-Conditioned Invocation Safety in Agent Systemscited by 3
  26. 2026SafeMCP: Proactive Power Regulation for LLM Agent Defense via Environment-Grounded Look-Ahead Reasoningcited by 3
  27. 2026Untrusted Content Masking for Web Agents with Security Guaranteescited by 3
  28. 2025Quantifying Conversation Drift in MCP via Latent Polytopecited by 2
  29. 2026Invisible Threats from Model Context Protocol: Generating Stealthy Injection Payload via Tree-based Adaptive Searchcited by 2
  30. 2026Don't Make Models Guess Security and Safety: Symbolic Guardrails for Domain-Specific AI Agentscited by 2
  31. 2026Proof-of-Execution Memory: Defending LLM Agents Against Forged-Reasoning Attacks by Verifying What Actually Happenedcited by 2
  32. 2026ROPE: Routed Origin Policy Enforcement against Indirect Prompt Injectioncited by 2
  33. 2026Causality Laundering: Denial-Feedback Leakage in Tool-Calling LLM Agentscited by 1
  34. 2026GIF: Locally Sound Geometric Information Flow Control for LLMscited by 1
  35. 2026Prismata: Confining Cross-Site Prompt Injection in Web Agentscited by 1
  36. 2026NEXUS: Structured Runtime Safety for Tool-Using LLM Agentscited by 1
  37. 2026ToolMinimize: Auditing and Rewriting LLM Agent Tool Calls to Minimize Privacy Exposurecited by 1
  38. 2026PIIGuard: Mitigating PII Harvesting under Adversarial Sanitizationcited by 0

50 most cited of 69 All 69 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Imprompter: Tricking LLM Agents into Improper Tool Use
  2. 2026AgentVisor: Defending LLM Agents Against Prompt Injection via Semantic Virtualization
  3. 2026TraceGrant: A Contract-Governed Security Framework for the Task-Effect Lifecycle of Networked LLM Agents
  4. 2026Silent Egress: When Implicit Prompt Injection Makes LLM Agents Leak Without a Trace
  5. 2024Whispers in the Machine: Confidentiality in Agentic Systems
  6. 2026SafeClawBench: Separating Semantic, Audit-Evidence, and Sandbox Harm in Tool-Using LLM Agents
  7. 2026ClawGuard: A Runtime Security Framework for Tool-Augmented LLM Agents Against Indirect Prompt Injection
  8. 2026ARGUS: Defending LLM Agents Against Context-Aware Prompt Injection