# RTBAS: Defending LLM Agents Against Prompt Injection and Privacy Leakage

arXiv: https://arxiv.org/abs/2502.08966 · HTML: https://mlsanity.com/paper/2502.08966/

## Key facts

- Cited by 69 papers in the map (data through 14 Oct 2026).
- Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use.
- Builds on 20 of its 46 references that are in the map.
- First posted on arXiv 13 Feb 2025.

## Placed in

- LLM agents and tool use / [Agent threats, guardrails and governance](https://mlsanity.com/agents/agent-threats-guardrails-and-governance/)
- LLM safety and security / [Prompt injection and agent security](https://mlsanity.com/safety/prompt-injection-and-agent-security/)

## Builds on (most cited first, 20 of 20)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [OPT: Open Pre-trained Transformer Language Models](https://mlsanity.com/paper/2205.01068/) (2022)
- [Extracting Training Data from Large Language Models](https://mlsanity.com/paper/2012.07805/) (2020)
- [Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone](https://mlsanity.com/paper/2404.14219/) (2024)
- [Attention is not Explanation](https://mlsanity.com/paper/1902.10186/) (2019)
- [H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models](https://mlsanity.com/paper/2306.14048/) (2023)
- [Are Sixteen Heads Really Better than One?](https://mlsanity.com/paper/1905.10650/) (2019)
- [AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models](https://mlsanity.com/paper/2310.04451/) (2023)
- [Attention is not not Explanation](https://mlsanity.com/paper/1908.04626/) (2019)
- [Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Models](https://mlsanity.com/paper/2305.04091/) (2023)
- [InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents](https://mlsanity.com/paper/2403.02691/) (2024)
- [Formalizing and Benchmarking Prompt Injection Attacks and Defenses](https://mlsanity.com/paper/2310.12815/) (2023)
- [Label Words are Anchors: An Information Flow Perspective for Understanding In-Context Learning](https://mlsanity.com/paper/2305.14160/) (2023)
- [AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents](https://mlsanity.com/paper/2406.13352/) (2024)
- [ProPILE: Probing Privacy Leakage in Large Language Models](https://mlsanity.com/paper/2307.01881/) (2023)
- [Attention Tracker: Detecting Prompt Injection Attacks in LLMs](https://mlsanity.com/paper/2411.00348/) (2024)
- [The Benefits of a Concise Chain of Thought on Problem-Solving in Large Language Models](https://mlsanity.com/paper/2401.05618/) (2024)
- [PAPILLON: Privacy Preservation from Internet-based and Local Language Model Ensembles](https://mlsanity.com/paper/2410.17127/) (2024)
- [CLEAR: Towards Contextual LLM-Empowered Privacy Policy Analysis and Risk Generation for Large Language Model Applications](https://mlsanity.com/paper/2410.13387/) (2024)
- [Fine-tuned Large Language Models (LLMs): Improved Prompt Injection Attacks Detection](https://mlsanity.com/paper/2410.21337/) (2024)

## Built on by (most cited first, 30 of 69)

- [Defeating Prompt Injections by Design](https://mlsanity.com/paper/2503.18813/) (2025)
- [Large Language Model Agent: A Survey on Methodology, Applications and Challenges](https://mlsanity.com/paper/2503.21460/) (2025)
- [Securing AI Agents with Information-Flow Control](https://mlsanity.com/paper/2505.23643/) (2025)
- [A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deployment](https://mlsanity.com/paper/2504.15585/) (2025)
- [A Survey of WebAgents: Towards Next-Generation AI Agents for Web Automation with Large Foundation Models](https://mlsanity.com/paper/2503.23350/) (2025)
- [DRIFT: Dynamic Rule-Based Defense with Injection Isolation for Securing LLM Agents](https://mlsanity.com/paper/2506.12104/) (2025)
- [AgentArmor: Enforcing Program Analysis on Agent Runtime Trace to Defend Against Prompt Injection](https://mlsanity.com/paper/2508.01249/) (2025)
- [Towards Verifiably Safe Tool Use for LLM Agents](https://mlsanity.com/paper/2601.08012/) (2026)
- [The Landscape of Prompt Injection Threats in LLM Agents: From Taxonomy to Analysis](https://mlsanity.com/paper/2602.10453/) (2026)
- [LLM Agents Should Employ Security Principles](https://mlsanity.com/paper/2505.24019/) (2025)
- [The Attack and Defense Landscape of Agentic AI: A Comprehensive Survey](https://mlsanity.com/paper/2603.11088/) (2026)
- [Simple Prompt Injection Attacks Can Leak Personal Data Observed by LLM Agents During Task Execution](https://mlsanity.com/paper/2506.01055/) (2025)
- [A Survey on the Safety and Security Threats of Computer-Using Agents: JARVIS or Ultron?](https://mlsanity.com/paper/2505.10924/) (2025)
- [TopicAttack: An Indirect Prompt Injection Attack via Topic Transition](https://mlsanity.com/paper/2507.13686/) (2025)
- [CaMeLs Can Use Computers Too: System-level Security for Computer Use Agents](https://mlsanity.com/paper/2601.09923/) (2026)
- [Robustness via Referencing: Defending against Prompt Injection Attacks by Referencing the Executed Instruction](https://mlsanity.com/paper/2504.20472/) (2025)
- [Ghost in the Agent: Redefining Information Flow Tracking for LLM Agents](https://mlsanity.com/paper/2604.23374/) (2026)
- [Aligning Provenance with Authorization: A Dual-Graph Defense for LLM Agents](https://mlsanity.com/paper/2605.26497/) (2026)
- [Systems Security Foundations for Agentic Computing](https://mlsanity.com/paper/2512.01295/) (2025)
- [AIRGuard: Guarding Agent Actions with Runtime Authority Control](https://mlsanity.com/paper/2605.28914/) (2026)
- [Position: The Real Barrier to LLM Agent Usability is Agentic ROI](https://mlsanity.com/paper/2505.17767/) (2025)
- [Secure and Efficient Access Control for Computer-Use Agents via Context Space](https://mlsanity.com/paper/2509.22256/) (2025)
- [AgentGuardian: Learning Access Control Policies to Govern AI Agent Behavior](https://mlsanity.com/paper/2601.10440/) (2026)
- [AgentSys: Secure and Dynamic LLM Agents Through Explicit Hierarchical Memory Management](https://mlsanity.com/paper/2602.07398/) (2026)
- [Optimizing Agent Planning for Security and Autonomy](https://mlsanity.com/paper/2602.11416/) (2026)
- [Cloak and Detonate: Scanner Evasion and Dynamic Detection of Agent Skill Malware](https://mlsanity.com/paper/2607.02357/) (2026)
- [Defense Against Indirect Prompt Injection via Tool Result Parsing](https://mlsanity.com/paper/2601.04795/) (2026)
- [The Promptware Kill Chain: How Prompt Injections Gradually Evolved Into a Multistep Malware Delivery Mechanism](https://mlsanity.com/paper/2601.09625/) (2026)
- [Reframing LLM Agent Security as an Agent-Human Interaction Problem](https://mlsanity.com/paper/2605.24309/) (2026)
- [Semia: Auditing Agent Skills via Constraint-Guided Representation Synthesis](https://mlsanity.com/paper/2605.00314/) (2026)

## Abstract (arXiv)

Tool-Based Agent Systems (TBAS) allow Language Models (LMs) to use external tools for tasks beyond their standalone capabilities, such as searching websites, booking flights, or making financial transactions. However, these tools greatly increase the risks of prompt injection attacks, where malicious content hijacks the LM agent to leak confidential data or trigger harmful actions. Existing defenses (OpenAI GPTs) require user confirmation before every tool call, placing onerous burdens on users. We introduce Robust TBAS (RTBAS), which automatically detects and executes tool calls that preserve integrity and confidentiality, requiring user confirmation only when these safeguards cannot be ensured. RTBAS adapts Information Flow Control to the unique challenges presented by TBAS. We present two novel dependency screeners, using LM-as-a-judge and attention-based saliency, to overcome these challenges. Experimental results on the AgentDojo Prompt Injection benchmark show RTBAS prevents all targeted attacks with only a 2% loss of task utility when under attack, and further tests confirm its ability to obtain near-oracle performance on detecting both subtle and direct privacy leaks.