mlsanityData to 7 Oct 2026

PromptShield: Deployable Detection for Prompt Injection Attacks

arXiv 2501.15145 ↗First posted 25 Jan 2025cs.CRcited by 43

On the map: LLM safety and security → Prompt injection and agent security

Abstract (arXiv)

Application designers have moved to integrate large language models (LLMs) into their products. However, many LLM-integrated applications are vulnerable to prompt injections. While attempts have been made to address this problem by building prompt injection detectors, many are not yet suitable for practical deployment. To support research in this area, we introduce PromptShield, a benchmark for training and evaluating deployable prompt injection detectors. Our benchmark is carefully curated and includes both conversational and application-structured data. In addition, we use insights from our curation process to fine-tune a new prompt injection detector that achieves significantly higher performance in the low false positive rate (FPR) evaluation regime compared to prior schemes. Our work suggests that careful curation of training data and larger models can contribute to strong detector performance.

Key facts. Cited by 43 papers in the map (data through 7 Oct 2026). Placed in Prompt injection and agent security, one of 12 directions in LLM safety and security. First posted on arXiv 25 Jan 2025.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 16; 2026: 27.

43 papers.

Sort:

By area:

  1. 2025PromptArmor: Simple yet Effective Prompt Injection Defensescited by 98
  2. 2025AgentArmor: Enforcing Program Analysis on Agent Runtime Trace to Defend Against Prompt Injectioncited by 39
  3. 2025Red Teaming the Mind of the Machine: A Systematic Evaluation of Prompt Injection and Jailbreak Vulnerabilities in LLMscited by 34
  4. 2026Towards Verifiably Safe Tool Use for LLM Agentscited by 34
  5. 2026The Attack and Defense Landscape of Agentic AI: A Comprehensive Surveycited by 26
  6. 2026MalTool: Malicious Tool Attacks on LLM Agentscited by 17
  7. 2026A Framework for Formalizing LLM Agent Securitycited by 16
  8. 2025Prompt Injection Detection and Mitigation via AI Multi-Agent NLP Frameworkscited by 14
  9. 2025To Protect the LLM Agent Against the Prompt Injection Attack with Polymorphic Promptcited by 14
  10. 2025PIShield: Detecting Prompt Injection Attacks via Intrinsic LLM Featurescited by 13
  11. 2026PISmith: Reinforcement Learning-based Red Teaming for Prompt Injection Defensescited by 13
  12. 2025CommandSans: Securing AI Agents with Surgical Precision Prompt Sanitizationcited by 12
31 more
  1. 2025Breaking and Fixing Defenses Against Control-Flow Hijacking in Multi-Agent Systemscited by 12
  2. 2025Preventing Prompt Injection with Type-Directed Privilege Separationcited by 8
  3. 2026When Benchmarks Lie: Evaluating Malicious Prompt Classifiers Under True Distribution Shiftcited by 8
  4. 2026Thought Virus: Viral Misalignment via Subliminal Prompting in Multi-Agent Systemscited by 7
  5. 2025Bypassing Prompt Guards in Production with Controlled-Release Promptingcited by 6
  6. 2025AttnTrace: Contextual Attribution of Prompt Injection and Knowledge Corruptioncited by 5
  7. 2025Evaluating the Robustness of Retrieval-Augmented Generation to Adversarial Evidence in the Health Domaincited by 5
  8. 2025CAPTURE: Context-Aware Prompt Injection Testing and Robustness Enhancementcited by 4
  9. 2026Prompt Injection Mitigation with Agentic AI, Nested Learning, and AI Sustainability via Semantic Cachingcited by 4
  10. 2025Token-Efficient Prompt Injection Attack: Provoking Cessation in LLM Reasoning via Adaptive Token Compressioncited by 3
  11. 2025A Framework for Rapidly Developing and Deploying Protection Against Large Language Model Attackscited by 3
  12. 2026Adversarial Attacks on Multimodal Large Language Models: A Comprehensive Surveycited by 3
  13. 2026From Prompt Injection to Persistent Control: Defending Agentic Harness Against Trojan Backdoorscited by 3
  14. 2026InjecMEM: Memory Injection Attack on LLM Agent Memory Systemscited by 3
  15. 2025Quantifying Conversation Drift in MCP via Latent Polytopecited by 2
  16. 2026Agent Privilege Separation in OpenClaw: A Structural Defense Against Prompt Injectioncited by 2
  17. 2026AgentDrift: A Step-Labeled Benchmark of Injection-Hijacked LLM Agent Trajectoriescited by 2
  18. 2026AI Security in the Foundation Model Era: A Comprehensive Survey from a Unified Perspectivecited by 1
  19. 2026Hallucination Mitigation with Agentic AI, Nested Learning, and AI Sustainability via Semantic Cachingcited by 1
  20. 2026PIDS-Bench: Evaluating Prompt-Injection Detectors Under Over-Defense, Obfuscation, and Distribution Shiftcited by 1
  21. 2026Light Alignment Improves LLM Safety via Model Self-Reflection with a Single Neuroncited by 0
  22. 2026Security Assessment and Mitigation Strategies for Large Language Models: A Comprehensive Defensive Frameworkcited by 0
  23. 2026Covert Visual Prompt Injection against Commercial Multimodal Large Language Modelscited by 0
  24. 2026Conversations Risk Detection LLMs in Financial Agents via Multi-Stage Generative Rolloutcited by 0
  25. 2026Cross-Agent Campaign Attribution: Linking Asynchronous Attacks Across LLM Agentscited by 0
  26. 2026Data Leakage Prevention in Agentic Applications via Preemptive Hardeningcited by 0
  27. 2026Twin Agent: Context Residual Compression for Privilege Separated Agentscited by 0
  28. 2026No-Box Vulnerability Analysis: Description-only Detection of Indirect Prompt Injection Vulnerabilities in MCP Serverscited by 0
  29. 2026DriftNet: A Dual-Head Trajectory Transformer for Detecting and Localizing Prompt Injection in LLM Agentscited by 0
  30. 2026Toward Secure AI-Powered Penetration Testing Agents: Security Threats, Guardrails, and Architectural Perspectivescited by 0
  31. 2026Prompt Injection Detection for Email Agents Through Attack Chain Modelingcited by 0