mlsanityData to 7 Oct 2026

Signed-Prompt: A New Approach to Prevent Prompt Injection Attacks Against LLM-Integrated Applications

arXiv 2401.07612 ↗First posted 15 Jan 2024cs.CR cs.AIcited by 42

On the map: LLM safety and security → Prompt injection and agent security

Abstract (arXiv)

The critical challenge of prompt injection attacks in Large Language Models (LLMs) integrated applications, a growing concern in the Artificial Intelligence (AI) field. Such attacks, which manipulate LLMs through natural language inputs, pose a significant threat to the security of these applications. Traditional defense strategies, including output and input filtering, as well as delimiter use, have proven inadequate. This paper introduces the 'Signed-Prompt' method as a novel solution. The study involves signing sensitive instructions within command segments by authorized users, enabling the LLM to discern trusted instruction sources. The paper presents a comprehensive analysis of prompt injection attack patterns, followed by a detailed explanation of the Signed-Prompt concept, including its basic architecture and implementation through both prompt engineering and fine-tuning of LLMs. Experiments demonstrate the effectiveness of the Signed-Prompt method, showing substantial resistance to various types of prompt injection attacks, thus validating its potential as a robust defense strategy in AI security.

Key facts. Cited by 42 papers in the map (data through 7 Oct 2026). Placed in Prompt injection and agent security, one of 12 directions in LLM safety and security. Builds on 9 of its 10 references that are in the map. First posted on arXiv 15 Jan 2024.

Builds on

9 of 10 references.

Closest ancestors · 5

Same direction, newest first

  1. 2023Assessing Prompt Injection Risks in 200+ Custom GPTs20 Nov
  2. 2023Tensor Trust: Interpretable Prompt Injection Attacks from an Online Game2 Nov
  3. 2023Formalizing and Benchmarking Prompt Injection Attacks and Defenses19 Oct
  4. 2023Prompt Injection attack against LLM-integrated Applications8 Jun
  5. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection23 Feb

Other references · 4

  1. 2022Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distributioncited by 722
  2. 2023Review of Large Vision Models and Visual Prompt Engineeringcited by 77
  3. 2022Evaluating the Susceptibility of Pre-Trained Language Models via Handcrafted Adversarial Examplescited by 65
  4. 2021Improved Regularization and Robustness for Fine-tuning in Neural Networkscited by 45

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 10; 2025: 14; 2026: 18.

42 papers.

Sort:

By area:

  1. 2024InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agentscited by 476
  2. 2024SecAlign: Defending Against Prompt Injection with Preference Optimizationcited by 179
  3. 2024Attention Tracker: Detecting Prompt Injection Attacks in LLMscited by 82
  4. 2025DRIFT: Dynamic Rule-Based Defense with Injection Isolation for Securing LLM Agentscited by 59
  5. 2025Multi-Agent Systems Execute Arbitrary Malicious Codecited by 52
  6. 2024Operationalizing a Threat Model for Red-Teaming Large Language Models (LLMs)cited by 44
  7. 2025Red Teaming the Mind of the Machine: A Systematic Evaluation of Prompt Injection and Jailbreak Vulnerabilities in LLMscited by 34
  8. 2024PrivacyRestore: Privacy-Preserving Inference in Large Language Models via Privacy Removal and Restorationcited by 27
  9. 2025Agentic AI Security: Threats, Defenses, Evaluation, and Open Challengescited by 26
  10. 2024GoEX: Perspectives and Designs Towards a Runtime for Autonomous LLM Applicationscited by 22
  11. 2025Prompt Injection Detection and Mitigation via AI Multi-Agent NLP Frameworkscited by 14
  12. 2025Attack and defense techniques in large language models: A survey and new perspectivescited by 13
30 more
  1. 2025LLM Security: Vulnerabilities, Attacks, Defenses, and Countermeasurescited by 8
  2. 2026Poise: Position-Aware One-Instruction Skill Injection for Silent Execution on LLM Agentscited by 7
  3. 2024Can LLMs be Fooled? Investigating Vulnerabilities in LLMscited by 5
  4. 2024Recent Advances in Attack and Defense Approaches of Large Language Modelscited by 5
  5. 2026Prompt Injection Mitigation with Agentic AI, Nested Learning, and AI Sustainability via Semantic Cachingcited by 4
  6. 2026SkillScope: Toward Fine-Grained Least-Privilege Enforcement for Agent Skillscited by 4
  7. 2024Palisade -- Prompt Injection Detection Frameworkcited by 3
  8. 2025A Few Words Can Distort Graphs: Knowledge Poisoning Attacks on Graph-based Retrieval-Augmented Generation of Large Language Modelscited by 3
  9. 2025Multi-Stage Prompt Inference Attacks on Enterprise LLM Systemscited by 2
  10. 2025Quantifying Conversation Drift in MCP via Latent Polytopecited by 2
  11. 2026Machine-Assisted Grading of Nationwide School-Leaving Essay Exams with LLMs and Statistical NLPcited by 2
  12. 2026Hijacking Large Audio-Language Models via Context-Agnostic and Imperceptible Auditory Prompt Injectioncited by 2
  13. 2024Bridging Today and the Future of Humanity: AI Safety in 2024 and Beyondcited by 1
  14. 2025An Empirically-grounded tool for Automatic Prompt Linting and Repair: A Case Study on Bias, Vulnerability, and Optimization in Developer Promptscited by 1
  15. 2025VSF-Med:A Vulnerability Scoring Framework for Medical Vision-Language Modelscited by 1
  16. 2025LLM in the Middle: A Systematic Review of Threats and Mitigations to Real-World LLM-based Systemscited by 1
  17. 2025CourtGuard: A Local, Multiagent Prompt Injection Classifiercited by 1
  18. 2026CASCADE: A Component Ablation and Corpus Audit of a Layered Local Defense for MCP-Based Systemscited by 1
  19. 2026What Does the Server See? Understanding Privacy Leakage from Large Language Models in Split Inferencecited by 1
  20. 2026Hallucination Mitigation with Agentic AI, Nested Learning, and AI Sustainability via Semantic Cachingcited by 1
  21. 2026Isolation as a First-Class Principle for LLM-Agent System Safety: Concepts, Taxonomy, Challenges and Future Directionscited by 1
  22. 2026BASIS: Breach-Aware Selective Prompt Injection Shielding with Prefill Attention Probescited by 1
  23. 2026Sirens' Whisper: Inaudible Near-Ultrasonic Jailbreaks of Speech-Driven LLMscited by 0
  24. 2026Security Assessment and Mitigation Strategies for Large Language Models: A Comprehensive Defensive Frameworkcited by 0
  25. 2026LogicPoison: Logical Attacks on Graph Retrieval-Augmented Generationcited by 0
  26. 2026When Prompts Become Payloads: A Framework for Mitigating SQL Injection Attacks in Large Language Model-Driven Applicationscited by 0
  27. 2026An Empirical Study of Privacy Leakage Chains via Prompt Injection in Black-Box Chatbot Environmentscited by 0
  28. 2026The safety failures we are not instrumenting: a perspective on hidden safety-critical challenges in modern AI systemscited by 0
  29. 2026PolicyGuard: Prompt-Configurable Semantic DLP for LLM Coding Agentscited by 0
  30. 2026Trust propagation and structural containment in Multi-agent LLM pipelinescited by 0