# Defending Against Prompt Injection with DataFilter

arXiv: https://arxiv.org/abs/2510.19207 · HTML: https://mlsanity.com/paper/2510.19207/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- Placed in Prompt injection and agent security, one of 12 directions in LLM safety and security.
- Builds on 37 of its 72 references that are in the map.
- First posted on arXiv 22 Oct 2025.

## Placed in

- LLM safety and security / [Prompt injection and agent security](https://mlsanity.com/safety/prompt-injection-and-agent-security/)

## Builds on (most cited first, 30 of 37)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Universal and Transferable Adversarial Attacks on Aligned Language Models](https://mlsanity.com/paper/2307.15043/) (2023)
- [Survey of Hallucination in Natural Language Generation](https://mlsanity.com/paper/2202.03629/) (2022)
- [Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection](https://mlsanity.com/paper/2302.12173/) (2023)
- [Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference](https://mlsanity.com/paper/2403.04132/) (2024)
- [Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluators](https://mlsanity.com/paper/2404.04475/) (2024)
- [Ignore Previous Prompt: Attack Techniques For Language Models](https://mlsanity.com/paper/2211.09527/) (2022)
- [AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedback](https://mlsanity.com/paper/2305.14387/) (2023)
- [InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents](https://mlsanity.com/paper/2403.02691/) (2024)
- [The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions](https://mlsanity.com/paper/2404.13208/) (2024)
- [Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrations](https://mlsanity.com/paper/2310.06387/) (2023)
- [StruQ: Defending Against Prompt Injection with Structured Queries](https://mlsanity.com/paper/2402.06363/) (2024)
- [Formalizing and Benchmarking Prompt Injection Attacks and Defenses](https://mlsanity.com/paper/2310.12815/) (2023)
- [Defeating Prompt Injections by Design](https://mlsanity.com/paper/2503.18813/) (2025)
- [Defending Against Indirect Prompt Injection Attacks With Spotlighting](https://mlsanity.com/paper/2403.14720/) (2024)
- [Benchmarking and Defending Against Indirect Prompt Injection Attacks on Large Language Models](https://mlsanity.com/paper/2312.14197/) (2023)
- [SecAlign: Defending Against Prompt Injection with Preference Optimization](https://mlsanity.com/paper/2410.05451/) (2024)
- [WASP: Benchmarking Web Agent Security Against Prompt Injection Attacks](https://mlsanity.com/paper/2504.18575/) (2025)
- [AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents](https://mlsanity.com/paper/2406.13352/) (2024)
- [DataSentinel: A Game-Theoretic Detection of Prompt Injection Attacks](https://mlsanity.com/paper/2504.11358/) (2025)
- [IsolateGPT: An Execution Isolation Architecture for LLM-Based Agentic Systems](https://mlsanity.com/paper/2403.04960/) (2024)
- [The Attacker Moves Second: Stronger Adaptive Attacks Bypass Defenses Against Llm Jailbreaks and Prompt Injections](https://mlsanity.com/paper/2510.09023/) (2025)
- [Automatic and Universal Prompt Injection Attacks against Large Language Models](https://mlsanity.com/paper/2403.04957/) (2024)
- [PromptArmor: Simple yet Effective Prompt Injection Defenses](https://mlsanity.com/paper/2507.15219/) (2025)
- [Neural Exec: Learning (and Learning from) Execution Triggers for Prompt Injection Attacks](https://mlsanity.com/paper/2403.03792/) (2024)
- [Can LLMs Separate Instructions From Data? And What Do We Even Mean By That?](https://mlsanity.com/paper/2403.06833/) (2024)
- [MELON: Provable Defense Against Indirect Prompt Injection Attacks in AI Agents](https://mlsanity.com/paper/2502.05174/) (2025)
- [Instructional Segment Embedding: Improving LLM Safety with Instruction Hierarchy](https://mlsanity.com/paper/2410.09102/) (2024)
- [Effectively Controlling Reasoning Models through Thinking Intervention](https://mlsanity.com/paper/2503.24370/) (2025)
- [IPIGuard: A Novel Tool Dependency Graph-Based Defense Against Indirect Prompt Injection in LLM Agents](https://mlsanity.com/paper/2508.15310/) (2025)

## Built on by (most cited first, 30 of 35)

- [From Untrusted Input to Trusted Memory: A Systematic Study of Memory Poisoning Attacks in LLM Agents](https://mlsanity.com/paper/2606.04329/) (2026)
- [Taming OpenClaw: Security Analysis and Mitigation of Autonomous LLM Agent Threats](https://mlsanity.com/paper/2603.11619/) (2026)
- [PISmith: Reinforcement Learning-based Red Teaming for Prompt Injection Defenses](https://mlsanity.com/paper/2603.13026/) (2026)
- [CausalArmor: Efficient Indirect Prompt Injection Guardrails via Causal Attribution](https://mlsanity.com/paper/2602.07918/) (2026)
- [A Systematic Security Evaluation of OpenClaw and Its Variants](https://mlsanity.com/paper/2604.03131/) (2026)
- [SoK: Trust-Authorization Mismatch in LLM Agent Interactions](https://mlsanity.com/paper/2512.06914/) (2025)
- [PIArena: A Platform for Prompt Injection Evaluation](https://mlsanity.com/paper/2604.08499/) (2026)
- [Assessing Automated Prompt Injection Attacks in Agentic Environments](https://mlsanity.com/paper/2606.10525/) (2026)
- [AgentWatcher: A Rule-based Prompt Injection Monitor](https://mlsanity.com/paper/2604.01194/) (2026)
- [When Agents Remember Too Much: Memory Poisoning Attacks on Large Language Model Agents](https://mlsanity.com/paper/2607.06595/) (2026)
- [AutoDojo: Adaptive Black-Box Attacks Reveal the Limits of IPI Defenses and Task-Specification Effects in LLM Agents](https://mlsanity.com/paper/2606.15057/) (2026)
- [InjecMEM: Memory Injection Attack on LLM Agent Memory Systems](https://mlsanity.com/paper/2608.23471/) (2026)
- [Agents at Risk: How Users Unwittingly Undermine LLM Safety](https://mlsanity.com/paper/2601.10758/) (2026)
- [Untrusted Content Masking for Web Agents with Security Guarantees](https://mlsanity.com/paper/2607.05277/) (2026)
- [SecOPD: Mitigating Adaptive Prompt Injections by On-Policy Distillation](https://mlsanity.com/paper/2608.21500/) (2026)
- [RedVisor: Reasoning-Aware Prompt Injection Defense via Zero-Copy KV Cache Reuse](https://mlsanity.com/paper/2602.01795/) (2026)
- [ROPE: Routed Origin Policy Enforcement against Indirect Prompt Injection](https://mlsanity.com/paper/2608.27496/) (2026)
- [Weight space Detection of Backdoors in LoRA Adapters](https://mlsanity.com/paper/2602.15195/) (2026)
- [LocalAlign: Enabling Generalizable Prompt Injection Defense via Generation of Near-Target Adversarial Examples for Alignment Training](https://mlsanity.com/paper/2605.01462/) (2026)
- [Agent Against Agent: An Agentic System for Automatic Prompt Injection Red Teaming](https://mlsanity.com/paper/2608.05108/) (2026)
- [What Guides the Agent? Adjudicating Unauthorized Behavior via Localizing Behavior-Guiding Instructions](https://mlsanity.com/paper/2608.24022/) (2026)
- [CLASP: Defending Hybrid Large Language Models Against Hidden State Poisoning Attacks](https://mlsanity.com/paper/2603.12206/) (2026)
- [Security Attack and Defense Strategies for Autonomous Agent Frameworks: A Layered Review with OpenClaw as a Case Study](https://mlsanity.com/paper/2604.27464/) (2026)
- [A Low-Latency Fraud Detection Layer for Detecting Adversarial Interaction Patterns in LLM-Powered Agents](https://mlsanity.com/paper/2605.01143/) (2026)
- [Ghost in the Context: Policy-Carriage Integrity in LLM Agents](https://mlsanity.com/paper/2605.12535/) (2026)
- [On the Geometric Limits of Transformer Defenses against Obfuscation Attacks: Latent Embedding Collapse & Performance Robustness Gap](https://mlsanity.com/paper/2605.19159/) (2026)
- [Rapid Poison: Practical Poisoning Attacks Against the Rapid Response Framework](https://mlsanity.com/paper/2606.16242/) (2026)
- [ALIBI: Adaptive Agentic Attacks on LLM-Based Vulnerability Detectors via Adversarial Code Comments](https://mlsanity.com/paper/2607.24964/) (2026)
- [On Understanding, Identifying, and Mitigating Vulnerabilities in Agentic Large Language Models](https://mlsanity.com/paper/2608.10530/) (2026)
- [PIPES: Securing Agent Perception with Provenance and Priors](https://mlsanity.com/paper/2608.12789/) (2026)

## Abstract (arXiv)

When large language model (LLM) agents are increasingly deployed to automate tasks and interact with untrusted external data, prompt injection emerges as a significant security threat. By injecting malicious instructions into the data that LLMs access, an attacker can arbitrarily override the original user task and redirect the agent toward unintended, potentially harmful actions. Existing defenses either require access to model weights (fine-tuning), incur substantial utility loss (detection-based), or demand non-trivial system redesign (system-level). Motivated by this, we propose DataFilter, a test-time model-agnostic defense that removes malicious instructions from the data before it reaches the backend LLM. DataFilter is trained with supervised fine-tuning on simulated injections and leverages both the user's instruction and the data to selectively strip adversarial content while preserving benign information. Across multiple benchmarks, DataFilter consistently reduces the prompt injection attack success rates to near zero while maintaining the LLMs' utility. DataFilter delivers strong security, high utility, and plug-and-play deployment, making it a strong practical defense to secure black-box commercial LLMs against prompt injection. Our DataFilter model is released at https://huggingface.co/JoyYizhu/DataFilter for immediate use, with the code to reproduce our results at https://github.com/yizhu-joy/DataFilter.