# Palisade -- Prompt Injection Detection Framework

arXiv: https://arxiv.org/abs/2410.21146 · HTML: https://mlsanity.com/paper/2410.21146/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Prompt injection and agent security, one of 12 directions in LLM safety and security.
- Builds on 12 of its 21 references that are in the map.
- First posted on arXiv 28 Oct 2024.

## Placed in

- LLM safety and security / [Prompt injection and agent security](https://mlsanity.com/safety/prompt-injection-and-agent-security/)

## Builds on (most cited first, 12 of 12)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [A Comprehensive Overview of Large Language Models](https://mlsanity.com/paper/2307.06435/) (2023)
- [How to Fine-Tune BERT for Text Classification?](https://mlsanity.com/paper/1905.05583/) (2019)
- [Benchmarking and Defending Against Indirect Prompt Injection Attacks on Large Language Models](https://mlsanity.com/paper/2312.14197/) (2023)
- [Use of LLMs for Illicit Purposes: Threats, Prevention Measures, and Vulnerabilities](https://mlsanity.com/paper/2308.12833/) (2023)
- [Signed-Prompt: A New Approach to Prevent Prompt Injection Attacks Against LLM-Integrated Applications](https://mlsanity.com/paper/2401.07612/) (2024)
- [An Early Categorization of Prompt Injection Attacks on Large Language Models](https://mlsanity.com/paper/2402.00898/) (2024)
- [Goal-guided Generative Prompt Injection Attack on Large Language Models](https://mlsanity.com/paper/2404.07234/) (2024)
- [Reasoning before Comparison: LLM-Enhanced Semantic Similarity Metrics for Domain Specialized Text Analysis](https://mlsanity.com/paper/2402.11398/) (2024)
- [Prompt Injection Attacks on Large Language Models in Oncology](https://mlsanity.com/paper/2407.18981/) (2024)
- [Applying Pre-trained Multilingual BERT in Embeddings for Improved Malicious Prompt Injection Attacks Detection](https://mlsanity.com/paper/2409.13331/) (2024)
- [Prompt Injection Attacks in Defended Systems](https://mlsanity.com/paper/2406.14048/) (2024)

## Built on by (most cited first, 5 of 5)

- [Red Teaming the Mind of the Machine: A Systematic Evaluation of Prompt Injection and Jailbreak Vulnerabilities in LLMs](https://mlsanity.com/paper/2505.04806/) (2025)
- [AgentSentry: Mitigating Indirect Prompt Injection in LLM Agents via Temporal Causal Diagnostics and Context Purification](https://mlsanity.com/paper/2602.22724/) (2026)
- [AegisMCP: Online Graph Intrusion Detection for Tool-Augmented LLMs on Edge Devices](https://mlsanity.com/paper/2510.19462/) (2025)
- [Proactive Hardening of LLM Defenses with HASTE](https://mlsanity.com/paper/2601.19051/) (2026)
- [RAG-PIBench: A Leakage-Aware Benchmark for Prompt-Injection Detection in Trustworthy RAG Systems](https://mlsanity.com/paper/2610.08571/) (2026)

## Abstract (arXiv)

The advent of Large Language Models LLMs marks a milestone in Artificial Intelligence, altering how machines comprehend and generate human language. However, LLMs are vulnerable to malicious prompt injection attacks, where crafted inputs manipulate the models behavior in unintended ways, compromising system integrity and causing incorrect outcomes. Conventional detection methods rely on static, rule-based approaches, which often fail against sophisticated threats like abnormal token sequences and alias substitutions, leading to limited adaptability and higher rates of false positives and false negatives.This paper proposes a novel NLP based approach for prompt injection detection, emphasizing accuracy and optimization through a layered input screening process. In this framework, prompts are filtered through three distinct layers rule-based, ML classifier, and companion LLM before reaching the target model, thereby minimizing the risk of malicious interaction.Tests show the ML classifier achieves the highest accuracy among individual layers, yet the multi-layer framework enhances overall detection accuracy by reducing false negatives. Although this increases false positives, it minimizes the risk of overlooking genuine injected prompts, thus prioritizing security.This multi-layered detection approach highlights LLM vulnerabilities and provides a comprehensive framework for future research, promoting secure interactions between humans and AI systems.