# Prompt Injection Detection and Mitigation via AI Multi-Agent NLP Frameworks

arXiv: https://arxiv.org/abs/2503.11517 · HTML: https://mlsanity.com/paper/2503.11517/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Placed in Prompt injection and agent security, one of 12 directions in LLM safety and security.
- Builds on 14 of its 21 references that are in the map.
- First posted on arXiv 14 Mar 2025.

## Placed in

- LLM safety and security / [Prompt injection and agent security](https://mlsanity.com/safety/prompt-injection-and-agent-security/)

## Builds on (most cited first, 14 of 14)

- [Universal Adversarial Triggers for Attacking and Analyzing NLP](https://mlsanity.com/paper/1908.07125/) (2019)
- [Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailment](https://mlsanity.com/paper/1907.11932/) (2019)
- [Detecting Language Model Attacks with Perplexity](https://mlsanity.com/paper/2308.14132/) (2023)
- [Automated Design of Agentic Systems](https://mlsanity.com/paper/2408.08435/) (2024)
- [MLLM-as-a-Judge: Assessing Multimodal LLM-as-a-Judge with Vision-Language Benchmark](https://mlsanity.com/paper/2402.04788/) (2024)
- [Formalizing and Benchmarking Prompt Injection Attacks and Defenses](https://mlsanity.com/paper/2310.12815/) (2023)
- [Jatmo: Prompt Injection Defense by Task-Specific Finetuning](https://mlsanity.com/paper/2312.17673/) (2023)
- [Signed-Prompt: A New Approach to Prevent Prompt Injection Attacks Against LLM-Integrated Applications](https://mlsanity.com/paper/2401.07612/) (2024)
- [AutoAgent: A Fully-Automated and Zero-Code Framework for LLM Agents](https://mlsanity.com/paper/2502.05957/) (2025)
- [PromptShield: Deployable Detection for Prompt Injection Attacks](https://mlsanity.com/paper/2501.15145/) (2025)
- [Backdoored Retrievers for Prompt Injection Attacks on Retrieval Augmented Generation of Large Language Models](https://mlsanity.com/paper/2410.14479/) (2024)
- [Hallucination Mitigation using Agentic AI Natural Language-Based Frameworks](https://mlsanity.com/paper/2501.13946/) (2025)
- [AI Multi-Agent Interoperability Extension for Managing Multiparty Conversations](https://mlsanity.com/paper/2411.05828/) (2024)
- [Exploring Potential Prompt Injection Attacks in Federated Military LLMs and Their Mitigation](https://mlsanity.com/paper/2501.18416/) (2025)

## Built on by (most cited first, 15 of 15)

- [AgentSentry: Mitigating Indirect Prompt Injection in LLM Agents via Temporal Causal Diagnostics and Context Purification](https://mlsanity.com/paper/2602.22724/) (2026)
- [Agentic AI Security: Threats, Defenses, Evaluation, and Open Challenges](https://mlsanity.com/paper/2510.23883/) (2025)
- [A Survey of Agentic AI and Cybersecurity: Challenges, Opportunities and Use-case Prototypes](https://mlsanity.com/paper/2601.05293/) (2026)
- [A Framework for Formalizing LLM Agent Security](https://mlsanity.com/paper/2603.19469/) (2026)
- [MedSentry: Understanding and Mitigating Safety Risks in Medical LLM Multi-Agent Systems](https://mlsanity.com/paper/2505.20824/) (2025)
- [Sentinel Agents for Secure and Trustworthy Agentic AI in Multi-Agent Systems](https://mlsanity.com/paper/2509.14956/) (2025)
- [From Chatbot to Digital Colleague: The Paradigm Shift Toward Persistent Autonomous AI](https://mlsanity.com/paper/2606.14502/) (2026)
- [Prompt Injection Mitigation with Agentic AI, Nested Learning, and AI Sustainability via Semantic Caching](https://mlsanity.com/paper/2601.13186/) (2026)
- [Counterfactual Evaluation for Blind Attack Detection in LLM-based Evaluation Systems](https://mlsanity.com/paper/2507.23453/) (2025)
- [Agentic AI Sustainability Assessment for Supply Chain Document Insights](https://mlsanity.com/paper/2511.07097/) (2025)
- [Zero-Shot Embedding Drift Detection: A Lightweight Defense Against Prompt Injections in LLMs](https://mlsanity.com/paper/2601.12359/) (2026)
- [CourtGuard: A Local, Multiagent Prompt Injection Classifier](https://mlsanity.com/paper/2510.19844/) (2025)
- [MADP: A Multi-Agent Pipeline for Sustainable Document Processing with Human-in-the-Loop](https://mlsanity.com/paper/2605.17159/) (2026)
- [Hallucination Mitigation with Agentic AI, Nested Learning, and AI Sustainability via Semantic Caching](https://mlsanity.com/paper/2605.29055/) (2026)
- [Cross-Agent Campaign Attribution: Linking Asynchronous Attacks Across LLM Agents](https://mlsanity.com/paper/2607.18826/) (2026)

## Abstract (arXiv)

Prompt injection constitutes a significant challenge for generative AI systems by inducing unintended outputs. We introduce a multi-agent NLP framework specifically designed to address prompt injection vulnerabilities through layered detection and enforcement mechanisms. The framework orchestrates specialized agents for generating responses, sanitizing outputs, and enforcing policy compliance. Evaluation on 500 engineered injection prompts demonstrates a marked reduction in injection success and policy breaches. Novel metrics, including Injection Success Rate (ISR), Policy Override Frequency (POF), Prompt Sanitization Rate (PSR), and Compliance Consistency Score (CCS), are proposed to derive a composite Total Injection Vulnerability Score (TIVS). The system utilizes the OVON (Open Voice Network) framework for inter-agent communication via structured JSON messages, extending a previously established multi-agent architecture from hallucination mitigation to address the unique challenges of prompt injection.