# Preventing Prompt Injection with Type-Directed Privilege Separation

arXiv: https://arxiv.org/abs/2509.25926 · HTML: https://mlsanity.com/paper/2509.25926/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use.
- Builds on 25 of its 45 references that are in the map.
- First posted on arXiv 30 Sep 2025.

## Placed in

- LLM agents and tool use / [Agent threats, guardrails and governance](https://mlsanity.com/agents/agent-threats-guardrails-and-governance/)
- LLM safety and security / [Prompt injection and agent security](https://mlsanity.com/safety/prompt-injection-and-agent-security/)

## Builds on (most cited first, 25 of 25)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities](https://mlsanity.com/paper/2507.06261/) (2025)
- [SWE-bench: Can Language Models Resolve Real-World GitHub Issues?](https://mlsanity.com/paper/2310.06770/) (2023)
- [Universal and Transferable Adversarial Attacks on Aligned Language Models](https://mlsanity.com/paper/2307.15043/) (2023)
- [SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering](https://mlsanity.com/paper/2405.15793/) (2024)
- [OpenAI Gym](https://mlsanity.com/paper/1606.01540/) (2016)
- [Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection](https://mlsanity.com/paper/2302.12173/) (2023)
- [WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents](https://mlsanity.com/paper/2207.01206/) (2022)
- [Ignore Previous Prompt: Attack Techniques For Language Models](https://mlsanity.com/paper/2211.09527/) (2022)
- [Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methods](https://mlsanity.com/paper/1705.07263/) (2017)
- ["Do Anything Now": Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Models](https://mlsanity.com/paper/2308.03825/) (2023)
- [Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrations](https://mlsanity.com/paper/2310.06387/) (2023)
- [StruQ: Defending Against Prompt Injection with Structured Queries](https://mlsanity.com/paper/2402.06363/) (2024)
- [Defeating Prompt Injections by Design](https://mlsanity.com/paper/2503.18813/) (2025)
- [SecAlign: Defending Against Prompt Injection with Preference Optimization](https://mlsanity.com/paper/2410.05451/) (2024)
- [IsolateGPT: An Execution Isolation Architecture for LLM-Based Agentic Systems](https://mlsanity.com/paper/2403.04960/) (2024)
- [Securing AI Agents with Information-Flow Control](https://mlsanity.com/paper/2505.23643/) (2025)
- [Jatmo: Prompt Injection Defense by Task-Specific Finetuning](https://mlsanity.com/paper/2312.17673/) (2023)
- [System-Level Defense against Indirect Prompt Injection Attacks: An Information Flow Control Perspective](https://mlsanity.com/paper/2409.19091/) (2024)
- [Prompt Flow Integrity to Prevent Privilege Escalation in LLM Agents](https://mlsanity.com/paper/2503.15547/) (2025)
- [Design Patterns for Securing LLM Agents against Prompt Injections](https://mlsanity.com/paper/2506.08837/) (2025)
- [PromptShield: Deployable Detection for Prompt Injection Attacks](https://mlsanity.com/paper/2501.15145/) (2025)
- [Tricking LLMs into Disobedience: Formalizing, Analyzing, and Detecting Jailbreaks](https://mlsanity.com/paper/2305.14965/) (2023)
- [JailbreaksOverTime: Detecting Jailbreak Attacks Under Distribution Shift](https://mlsanity.com/paper/2504.19440/) (2025)

## Built on by (most cited first, 9 of 9)

- [A Survey on Long-Term Memory Security in LLM Agents: Attacks, Defenses, and Governance Across the Memory Lifecycle](https://mlsanity.com/paper/2604.16548/) (2026)
- [Assessing Automated Prompt Injection Attacks in Agentic Environments](https://mlsanity.com/paper/2606.10525/) (2026)
- [Toward Secure LLM Agents: Threat Surfaces, Attacks, Defenses, and Evaluation](https://mlsanity.com/paper/2606.10749/) (2026)
- [Twin Agent: Context Residual Compression for Privilege Separated Agents](https://mlsanity.com/paper/2607.19595/) (2026)
- [PIPES: Securing Agent Perception with Provenance and Priors](https://mlsanity.com/paper/2608.12789/) (2026)
- [If Agents Were Angels, No Governance Would Be Necessary: Out-of-Band Policy Enforcement at a Trusted Tool Boundary](https://mlsanity.com/paper/2608.27646/) (2026)
- [A Proposal for an Agentic AI Architecture to Support Multi-Domain Decision-Making in the Brazilian Armed Forces](https://mlsanity.com/paper/2609.20080/) (2026)
- [Secure-CUA: Controlling Untrusted Influence in Computer-Use Agents](https://mlsanity.com/paper/2610.09469/) (2026)
- [Kairos: Transactional Security Enforcement at the System Boundary for LLM Agents](https://mlsanity.com/paper/2610.18457/) (2026)

## Abstract (arXiv)

Modern language models have enabled the development of agentic systems that achieve strong performance on reasoning-intensive tasks. Unfortunately, this has come with a security cost; these systems are vulnerable to prompt injection, a specialized attack where an adversary subverts the intended functionality of an agent by supplying an injected task of their own. Previous approaches address this challenge with detectors and fine-tuning defenses but are vulnerable to adaptive attacks. Other methods propose system-level defenses that guarantee security, but these are often based on techniques that prevent inter-component communication and thus are constrained in problem coverage. To this end, we introduce type-directed privilege separation, a new technique that expands the set of tasks that can be protected with system-level defenses. Our method works by converting untrusted data to a curated set of data types; unlike raw strings, each data type is limited in scope and content, eliminating the possibility for prompt injection. We evaluate our method across several case studies and find that designs using our principles can systematically prevent prompt injection attacks while featuring strong, non-trivial utility. Our approach is intuitive to understand and compatible with any language model.