The Promptware Kill Chain: How Prompt Injections Gradually Evolved Into a Multistep Malware Delivery Mechanism
On the map: LLM safety and security → Prompt injection and agent security
Abstract (arXiv)
Prompt injection was initially framed as the large language model (LLM) analogue of SQL injection. However, over the past three years, attacks labeled as prompt injection have evolved from isolated input-manipulation exploits into multistep attack mechanisms that resemble malware. In this paper, we argue that prompt injections evolved into promptware, a new class of malware execution mechanism triggered through prompts engineered to exploit an application's LLM. We introduce a seven-stage promptware kill chain: Initial Access (prompt injection), Privilege Escalation (jailbreaking), Reconnaissance, Persistence (memory and retrieval poisoning), Command and Control, Lateral Movement, and Actions on Objective. We analyze thirty-six prominent studies and real-world incidents affecting production LLM systems and show that at least twenty-one documented attacks that traverse four or more stages of this kill chain, demonstrating that the threat model is not merely theoretical. We discuss the need for a defense-in-depth approach that addresses all stages of the promptware life cycle and review relevant countermeasures for each step. By moving the conversation from prompt injection to a promptware kill chain, our work provides analytical clarity, enables structured risk assessment, and lays a foundation for systematic security engineering of LLM-based systems.
Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Placed in Prompt injection and agent security, one of 12 directions in LLM safety and security. Builds on 46 of its 99 references that are in the map. First posted on arXiv 14 Jan 2026.
Builds on
46 of 99 references.
Closest ancestors · 17
Same direction, newest first
- 2025Prompt Fencing: A Cryptographic Approach to Establishing Security Boundaries in Large Language Model Prompts24 Nov
- 2025Defending Against Prompt Injection With a Few DefensiveTokens10 Jul
- 2025Design Patterns for Securing LLM Agents against Prompt Injections10 Jun
- 2025A Critical Evaluation of Defenses against Prompt Injection Attacks23 May
- 2025Defeating Prompt Injections by Design24 Mar
- 2025ASIDE: Architectural Separation of Instructions and Data in Language Models13 Mar
- 2025RTBAS: Defending LLM Agents Against Prompt Injection and Privacy Leakage13 Feb
- 2024Trust No AI: Prompt Injection Along The CIA Security Triad8 Dec
- 2024Defense Against Prompt Injection Attack by Leveraging Attack Techniques1 Nov
- 2024Instructional Segment Embedding: Improving LLM Safety with Instruction Hierarchy9 Oct
- 2024Prompt Infection: LLM-to-LLM Prompt Injection within Multi-Agent Systems9 Oct
- 2024Get my drift? Catching LLM Task Drift with Activation Deltas2 Jun
5 more
- 2024The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions19 Apr
- 2024Defending Against Indirect Prompt Injection Attacks With Spotlighting20 Mar
- 2024StruQ: Defending Against Prompt Injection with Structured Queries9 Feb
- 2023Jatmo: Prompt Injection Defense by Task-Specific Finetuning29 Dec
- 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection23 Feb
Other references · 29
- 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
- 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
- 2022Ignore Previous Prompt: Attack Techniques For Language Modelscited by 760
- 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Modelscited by 608
- 2023"Do Anything Now": Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Modelscited by 457
- 2023SmoothLLM: Defending Large Language Models Against Jailbreaking Attackscited by 434
- 2023FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Promptscited by 368
- 2024Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attackcited by 364
- 2023NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable Railscited by 347
- 2023Are aligned neural networks adversarially aligned?cited by 305
- 2024ArtPrompt: ASCII Art-based Jailbreak Attacks against Aligned LLMscited by 197
- 2024AutoDefense: Multi-Agent LLM Defense against Jailbreak Attackscited by 135
17 more
- 2024IsolateGPT: An Execution Isolation Architecture for LLM-Based Agentic Systemscited by 118
- 2023Abusing Images and Sounds for Indirect Instruction Injection in Multi-Modal LLMscited by 117
- 2024AirGapAgent: Protecting Privacy-Conscious Conversational Agentscited by 90
- 2024JailbreakRadar: Comprehensive Assessment of Jailbreak Attacks Against LLMscited by 87
- 2024Defending LLMs against Jailbreaking Attacks via Backtranslationcited by 65
- 2025ACE: A Security Architecture for LLM-Integrated App Systemscited by 52
- 2024Defensive Prompt Patch: A Robust and Interpretable Defense of LLMs against Jailbreak Attackscited by 35
- 2025Firewalls to Secure Dynamic LLM Agentic Networkscited by 34
- 2023Adversarial Illusions in Multi-Modal Embeddingscited by 28
- 2024Protecting Your LLMs with Information Bottleneckcited by 26
- 2025AgentSentinel: An End-to-End and Real-Time Security Defense Framework for Computer-Use Agentscited by 25
- 2025SoK: Evaluating Jailbreak Guardrails for Large Language Modelscited by 17
- 2025Invitation Is All You Need! Promptware Attacks Against LLM-Powered Assistants in Production Are Practical and Dangerouscited by 11
- 2024A Jailbroken GenAI Model Can Cause Substantial Harm: GenAI-powered Applications are Vulnerable to PromptWarescited by 6
- 2025RAGuard: A Novel Approach for in-context Safe Retrieval Augmented Generation for LLMscited by 5
- 2025SoK: Systematizing LLM Prompt Security: Taxonomies, Datasets, and Unified Evaluation of Attacks and Defensescited by 4
- 2025ADA: Automated Moving Target Defense for AI Workloads via Ephemeral Infrastructure-Native Rotation in Kubernetescited by 1
Built on by
Citing papers per year: 2026: 7.
7 papers.
- 2026Oracle Poisoning: Corrupting Knowledge Graphs to Weaponise AI Agent Reasoningcited by 1
- 2026The Anatomy of a Prompt Injection: A Component Model for Structured Analysiscited by 0
- 2026Compositional Threat Analysis of Latent Compromise in LLM Agent Systems: The Order 66 Scenariocited by 0
- 2026TrustShiftProbe: Characterizing, Benchmarking, and Defending Staged Trust Attacks on MCP Serverscited by 0
- 2026Architecting the Secure AI-SOC: A Neurosymbolic Framework for Pipeline Integrity and Threat Mitigationcited by 0
- 2026Measuring and Exploiting Implicit Trust in LLM Tool-Calling Pipelinescited by 0
- 2026Beyond Single-Model Injection: A Threat Model and Defense Architecture for Prompt Injection in Multi-Agent Systemscited by 0
- 2026Beyond Single-Model Injection: A Threat Model and Defense Architecture for Prompt Injection in Multi-Agent Systems19 Sep 2026
- 2026Measuring and Exploiting Implicit Trust in LLM Tool-Calling Pipelines16 Sep 2026
- 2026Architecting the Secure AI-SOC: A Neurosymbolic Framework for Pipeline Integrity and Threat Mitigation9 Sep 2026
- 2026TrustShiftProbe: Characterizing, Benchmarking, and Defending Staged Trust Attacks on MCP Servers24 Aug 2026
- 2026Compositional Threat Analysis of Latent Compromise in LLM Agent Systems: The Order 66 Scenario8 Aug 2026
- 2026The Anatomy of a Prompt Injection: A Component Model for Structured Analysis7 Aug 2026
- 2026Oracle Poisoning: Corrupting Knowledge Graphs to Weaponise AI Agent Reasoning10 May 2026