mlsanityData to 14 Oct 2026

Prompt Injection Detection and Mitigation via AI Multi-Agent NLP Frameworks

arXiv 2503.11517 ↗First posted 14 Mar 2025cs.AI cs.CL cs.MAcited by 15

On the map: LLM safety and security → Prompt injection and agent security

Abstract (arXiv)

Prompt injection constitutes a significant challenge for generative AI systems by inducing unintended outputs. We introduce a multi-agent NLP framework specifically designed to address prompt injection vulnerabilities through layered detection and enforcement mechanisms. The framework orchestrates specialized agents for generating responses, sanitizing outputs, and enforcing policy compliance. Evaluation on 500 engineered injection prompts demonstrates a marked reduction in injection success and policy breaches. Novel metrics, including Injection Success Rate (ISR), Policy Override Frequency (POF), Prompt Sanitization Rate (PSR), and Compliance Consistency Score (CCS), are proposed to derive a composite Total Injection Vulnerability Score (TIVS). The system utilizes the OVON (Open Voice Network) framework for inter-agent communication via structured JSON messages, extending a previously established multi-agent architecture from hallucination mitigation to address the unique challenges of prompt injection.

Key facts. Cited by 15 papers in the map (data through 14 Oct 2026). Placed in Prompt injection and agent security, one of 12 directions in LLM safety and security. Builds on 14 of its 21 references that are in the map. First posted on arXiv 14 Mar 2025.

Builds on

14 of 21 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025Exploring Potential Prompt Injection Attacks in Federated Military LLMs and Their Mitigation30 Jan
  2. 2025PromptShield: Deployable Detection for Prompt Injection Attacks25 Jan
  3. 2024Signed-Prompt: A New Approach to Prevent Prompt Injection Attacks Against LLM-Integrated Applications15 Jan
  4. 2023Jatmo: Prompt Injection Defense by Task-Specific Finetuning29 Dec
  5. 2023Formalizing and Benchmarking Prompt Injection Attacks and Defenses19 Oct

Other references · 9

  1. 2019Universal Adversarial Triggers for Attacking and Analyzing NLPcited by 722
  2. 2019Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailmentcited by 671
  3. 2023Detecting Language Model Attacks with Perplexitycited by 431
  4. 2024Automated Design of Agentic Systemscited by 407
  5. 2024MLLM-as-a-Judge: Assessing Multimodal LLM-as-a-Judge with Vision-Language Benchmarkcited by 372
  6. 2025AutoAgent: A Fully-Automated and Zero-Code Framework for LLM Agentscited by 52
  7. 2024Backdoored Retrievers for Prompt Injection Attacks on Retrieval Augmented Generation of Large Language Modelscited by 22
  8. 2025Hallucination Mitigation using Agentic AI Natural Language-Based Frameworkscited by 12
  9. 2024AI Multi-Agent Interoperability Extension for Managing Multiparty Conversationscited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 6; 2026: 9.

15 papers.

Sort:

By area:

  1. 2026AgentSentry: Mitigating Indirect Prompt Injection in LLM Agents via Temporal Causal Diagnostics and Context Purificationcited by 29
  2. 2025Agentic AI Security: Threats, Defenses, Evaluation, and Open Challengescited by 28
  3. 2026A Survey of Agentic AI and Cybersecurity: Challenges, Opportunities and Use-case Prototypescited by 22
  4. 2026A Framework for Formalizing LLM Agent Securitycited by 16
  5. 2025MedSentry: Understanding and Mitigating Safety Risks in Medical LLM Multi-Agent Systemscited by 9
  6. 2025Sentinel Agents for Secure and Trustworthy Agentic AI in Multi-Agent Systemscited by 8
  7. 2026From Chatbot to Digital Colleague: The Paradigm Shift Toward Persistent Autonomous AIcited by 6
  8. 2026Prompt Injection Mitigation with Agentic AI, Nested Learning, and AI Sustainability via Semantic Cachingcited by 4
  9. 2025Counterfactual Evaluation for Blind Attack Detection in LLM-based Evaluation Systemscited by 3
  10. 2025Agentic AI Sustainability Assessment for Supply Chain Document Insightscited by 2
  11. 2026Zero-Shot Embedding Drift Detection: A Lightweight Defense Against Prompt Injections in LLMscited by 2
  12. 2025CourtGuard: A Local, Multiagent Prompt Injection Classifiercited by 1
3 more
  1. 2026MADP: A Multi-Agent Pipeline for Sustainable Document Processing with Human-in-the-Loopcited by 1
  2. 2026Hallucination Mitigation with Agentic AI, Nested Learning, and AI Sustainability via Semantic Cachingcited by 1
  3. 2026Cross-Agent Campaign Attribution: Linking Asynchronous Attacks Across LLM Agentscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Design Patterns for Securing LLM Agents against Prompt Injections
  2. 2026When Prompts Control Robots: Prompt Injection Attacks in Multi-Agent Robotic Systems
  3. 2026Beyond Single-Model Injection: A Threat Model and Defense Architecture for Prompt Injection in Multi-Agent Systems
  4. 2026The Landscape of Prompt Injection Threats in LLM Agents: From Taxonomy to Analysis
  5. 2024An Early Categorization of Prompt Injection Attacks on Large Language Models
  6. 2025PromptSleuth: Detecting Prompt Injection via Semantic Intent Invariance
  7. 2026Architecting Secure AI Agents: Perspectives on System-Level Defenses Against Indirect Prompt Injection Attacks
  8. 2025A Multi-Agent LLM Defense Pipeline Against Prompt Injection Attacks