# Signed-Prompt: A New Approach to Prevent Prompt Injection Attacks Against LLM-Integrated Applications

arXiv: https://arxiv.org/abs/2401.07612 · HTML: https://mlsanity.com/paper/2401.07612/

## Key facts

- Cited by 54 papers in the map (data through 14 Oct 2026).
- Placed in Prompt injection and agent security, one of 12 directions in LLM safety and security.
- Builds on 9 of its 10 references that are in the map.
- First posted on arXiv 15 Jan 2024.

## Placed in

- LLM safety and security / [Prompt injection and agent security](https://mlsanity.com/safety/prompt-injection-and-agent-security/)

## Builds on (most cited first, 9 of 9)

- [Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection](https://mlsanity.com/paper/2302.12173/) (2023)
- [Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distribution](https://mlsanity.com/paper/2202.10054/) (2022)
- [Prompt Injection attack against LLM-integrated Applications](https://mlsanity.com/paper/2306.05499/) (2023)
- [Formalizing and Benchmarking Prompt Injection Attacks and Defenses](https://mlsanity.com/paper/2310.12815/) (2023)
- [Tensor Trust: Interpretable Prompt Injection Attacks from an Online Game](https://mlsanity.com/paper/2311.01011/) (2023)
- [Review of Large Vision Models and Visual Prompt Engineering](https://mlsanity.com/paper/2307.00855/) (2023)
- [Evaluating the Susceptibility of Pre-Trained Language Models via Handcrafted Adversarial Examples](https://mlsanity.com/paper/2209.02128/) (2022)
- [Assessing Prompt Injection Risks in 200+ Custom GPTs](https://mlsanity.com/paper/2311.11538/) (2023)
- [Improved Regularization and Robustness for Fine-tuning in Neural Networks](https://mlsanity.com/paper/2111.04578/) (2021)

## Built on by (most cited first, 30 of 54)

- [InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents](https://mlsanity.com/paper/2403.02691/) (2024)
- [StruQ: Defending Against Prompt Injection with Structured Queries](https://mlsanity.com/paper/2402.06363/) (2024)
- [SecAlign: Defending Against Prompt Injection with Preference Optimization](https://mlsanity.com/paper/2410.05451/) (2024)
- [System-Level Defense against Indirect Prompt Injection Attacks: An Information Flow Control Perspective](https://mlsanity.com/paper/2409.19091/) (2024)
- [Attention Tracker: Detecting Prompt Injection Attacks in LLMs](https://mlsanity.com/paper/2411.00348/) (2024)
- [DRIFT: Dynamic Rule-Based Defense with Injection Isolation for Securing LLM Agents](https://mlsanity.com/paper/2506.12104/) (2025)
- [Multi-Agent Systems Execute Arbitrary Malicious Code](https://mlsanity.com/paper/2503.12188/) (2025)
- [Can Indirect Prompt Injection Attacks Be Detected and Removed?](https://mlsanity.com/paper/2502.16580/) (2025)
- [Operationalizing a Threat Model for Red-Teaming Large Language Models (LLMs)](https://mlsanity.com/paper/2407.14937/) (2024)
- [Defense Against Prompt Injection Attack by Leveraging Attack Techniques](https://mlsanity.com/paper/2411.00459/) (2024)
- [Red Teaming the Mind of the Machine: A Systematic Evaluation of Prompt Injection and Jailbreak Vulnerabilities in LLMs](https://mlsanity.com/paper/2505.04806/) (2025)
- [PrivacyRestore: Privacy-Preserving Inference in Large Language Models via Privacy Removal and Restoration](https://mlsanity.com/paper/2406.01394/) (2024)
- [Agentic AI Security: Threats, Defenses, Evaluation, and Open Challenges](https://mlsanity.com/paper/2510.23883/) (2025)
- [GoEX: Perspectives and Designs Towards a Runtime for Autonomous LLM Applications](https://mlsanity.com/paper/2404.06921/) (2024)
- [TopicAttack: An Indirect Prompt Injection Attack via Topic Transition](https://mlsanity.com/paper/2507.13686/) (2025)
- [Robustness via Referencing: Defending against Prompt Injection Attacks by Referencing the Executed Instruction](https://mlsanity.com/paper/2504.20472/) (2025)
- [Prompt Injection Detection and Mitigation via AI Multi-Agent NLP Frameworks](https://mlsanity.com/paper/2503.11517/) (2025)
- [Attack and defense techniques in large language models: A survey and new perspectives](https://mlsanity.com/paper/2505.00976/) (2025)
- [Evaluating LLM-based Personal Information Extraction and Countermeasures](https://mlsanity.com/paper/2408.07291/) (2024)
- [Attention is All You Need to Defend Against Indirect Prompt Injection Attacks in LLMs](https://mlsanity.com/paper/2512.08417/) (2025)
- [LLM Security: Vulnerabilities, Attacks, Defenses, and Countermeasures](https://mlsanity.com/paper/2505.01177/) (2025)
- [PromptSleuth: Detecting Prompt Injection via Semantic Intent Invariance](https://mlsanity.com/paper/2508.20890/) (2025)
- [Poise: Position-Aware One-Instruction Skill Injection for Silent Execution on LLM Agents](https://mlsanity.com/paper/2606.07943/) (2026)
- [Recent Advances in Attack and Defense Approaches of Large Language Models](https://mlsanity.com/paper/2409.03274/) (2024)
- [Can LLMs be Fooled? Investigating Vulnerabilities in LLMs](https://mlsanity.com/paper/2407.20529/) (2024)
- [Palisade -- Prompt Injection Detection Framework](https://mlsanity.com/paper/2410.21146/) (2024)
- [SkillScope: Toward Fine-Grained Least-Privilege Enforcement for Agent Skills](https://mlsanity.com/paper/2605.05868/) (2026)
- [A Few Words Can Distort Graphs: Knowledge Poisoning Attacks on Graph-based Retrieval-Augmented Generation of Large Language Models](https://mlsanity.com/paper/2508.04276/) (2025)
- [Prompt Injection Mitigation with Agentic AI, Nested Learning, and AI Sustainability via Semantic Caching](https://mlsanity.com/paper/2601.13186/) (2026)
- [What Does the Server See? Understanding Privacy Leakage from Large Language Models in Split Inference](https://mlsanity.com/paper/2605.23158/) (2026)

## Abstract (arXiv)

The critical challenge of prompt injection attacks in Large Language Models (LLMs) integrated applications, a growing concern in the Artificial Intelligence (AI) field. Such attacks, which manipulate LLMs through natural language inputs, pose a significant threat to the security of these applications. Traditional defense strategies, including output and input filtering, as well as delimiter use, have proven inadequate. This paper introduces the 'Signed-Prompt' method as a novel solution. The study involves signing sensitive instructions within command segments by authorized users, enabling the LLM to discern trusted instruction sources. The paper presents a comprehensive analysis of prompt injection attack patterns, followed by a detailed explanation of the Signed-Prompt concept, including its basic architecture and implementation through both prompt engineering and fine-tuning of LLMs. Experiments demonstrate the effectiveness of the Signed-Prompt method, showing substantial resistance to various types of prompt injection attacks, thus validating its potential as a robust defense strategy in AI security.