mlsanityData to 7 Oct 2026

Here Comes The AI Worm: Unleashing Zero-click Worms that Target GenAI-Powered Applications

arXiv 2403.02817 ↗First posted 5 Mar 2024cs.CRcited by 58

On the map: LLM safety and security → Prompt injection and agent security

Abstract (arXiv)

In this paper, we show that when the communication between GenAI-powered applications relies on RAG-based inference, an attacker can initiate a computer worm-like chain reaction that we call Morris-II. This is done by crafting an adversarial self-replicating prompt that triggers a cascade of indirect prompt injections within the ecosystem and forces each affected application to perform malicious actions and compromise the RAG of additional applications. We evaluate the performance of the worm in creating a chain of confidential user data extraction within a GenAI ecosystem of GenAI-powered email assistants and analyze how the performance of the worm is affected by the size of the context, the adversarial self-replicating prompt used, the type and size of the embedding algorithm employed, and the number of hops in the propagation. Finally, we introduce the Virtual Donkey, a guardrail intended to detect and prevent the propagation of Morris-II with minimal latency, high accuracy, and a low false-positive rate. We evaluate the guardrail's performance and show that it yields a perfect true-positive rate of 1.0 with a false-positive rate of 0.015, and is robust against out-of-distribution worms, consisting of unseen jailbreaking commands, a different email dataset, and various worm usecases.

Key facts. Cited by 58 papers in the map (data through 7 Oct 2026). Placed in Prompt injection and agent security, one of 12 directions in LLM safety and security. First posted on arXiv 5 Mar 2024.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 13; 2025: 21; 2026: 24.

58 papers.

Sort:

By area:

  1. 2024Prompt Infection: LLM-to-LLM Prompt Injection within Multi-Agent Systemscited by 126
  2. 2024AI Agents Under Threat: A Survey of Key Security Challenges and Future Pathwayscited by 122
  3. 2024Navigating the Risks: A Survey of Security, Privacy, and Ethics Threats in LLM-Based Agentscited by 63
  4. 2025Safety at Scale: A Comprehensive Survey of Large Model and Agent Safetycited by 63
  5. 2025A Survey of LLM-Driven AI Agent Communication: Protocols, Security Risks, and Defense Countermeasurescited by 54
  6. 2025Multi-Agent Systems Execute Arbitrary Malicious Codecited by 52
  7. 2025SAGA: A Security Architecture for Governing AI Agentic Systemscited by 45
  8. 2024Operationalizing a Threat Model for Red-Teaming Large Language Models (LLMs)cited by 44
  9. 2025Infrastructure for AI Agentscited by 40
  10. 2025AgentSafe: Safeguarding Large Language Model-based Multi-agent Systems via Hierarchical Data Managementcited by 35
  11. 2025GUARDIAN: Safeguarding LLM Multi-Agent Collaborations with Temporal Graph Modelingcited by 35
  12. 2024NetSafe: Exploring the Topological Safety of Multi-agent Networkscited by 29
38 more
  1. 2025LLM Agents Should Employ Security Principlescited by 28
  2. 2025Multi-Agent Security Tax: Trading Off Security and Collaboration Capabilities in Multi-Agent Systemscited by 27
  3. 2025Security Concerns for Large Language Models: A Surveycited by 26
  4. 2025Agentic AI Security: Threats, Defenses, Evaluation, and Open Challengescited by 26
  5. 2026The Landscape of Prompt Injection Threats in LLM Agents: From Taxonomy to Analysiscited by 25
  6. 2024Manipulating Large Language Models to Increase Product Visibilitycited by 24
  7. 2025Security Challenges in AI Agent Deployment: Insights from a Large Scale Public Competitioncited by 22
  8. 2026A Survey on Long-Term Memory Security in LLM Agents: Attacks, Defenses, and Governance Across the Memory Lifecyclecited by 21
  9. 2024IDs for AI Systemscited by 18
  10. 2026Overcoming the Retrieval Barrier: Indirect Prompt Injection in the Wild for LLM Systemscited by 16
  11. 2026Intelligent AI Delegationcited by 16
  12. 2026How Vulnerable Are AI Agents to Indirect Prompt Injections? Insights from a Large-Scale Public Competitioncited by 16
  13. 2026AgentWorm: Self-Propagating Attacks Across LLM Agent Ecosystemscited by 15
  14. 2025Prompt Injection 2.0: Hybrid AI Threatscited by 12
  15. 2025IP Leakage Attacks Targeting LLM-Based Multi-Agent Systemscited by 8
  16. 2026SOK: A Taxonomy of Attack Vectors and Defense Strategies for Agentic Supply Chain Runtimecited by 8
  17. 2025TAI3: Testing Agent Integrity in Interpreting User Intentcited by 7
  18. 2026SoK: Agentic Retrieval-Augmented Generation (RAG): Taxonomy, Architectures, Evaluation, and Research Directionscited by 7
  19. 2024A Jailbroken GenAI Model Can Cause Substantial Harm: GenAI-powered Applications are Vulnerable to PromptWarescited by 6
  20. 2026Safety in Self-Evolving LLM Agent Systems: Threats, Amplification, and Case Studiescited by 6
  21. 2026Autonomous LLM Agent Worms: Cross-Platform Propagation, Automated Discovery and Temporal Re-Entry Defensecited by 5
  22. 2026Towards trustworthy agentic AI: a comprehensive survey of safety, robustness, privacy, and system securitycited by 5
  23. 2026Governance Decay: How Context Compaction Silently Erases Safety Constraints in Long-Horizon LLM Agentscited by 5
  24. 2024Towards Action Hijacking of Large Language Model-based Agentcited by 4
  25. 2026State Contamination in Memory-Augmented LLM Agentscited by 4
  26. 2026Cross-Session Threats in AI Agents: Benchmark, Evaluation, and Algorithmscited by 3
  27. 2026Mind Viruses: Self-Propagating Ideas in Multi-Agent LLM Systemscited by 3
  28. 2024Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systemscited by 2
  29. 2025LogicLearner: A Tool for the Guided Practice of Propositional Logic Proofscited by 2
  30. 2025Technical Requirements for Halting Dangerous AI Activitiescited by 2
  31. 2024Countering Autonomous Cyber Threatscited by 1
  32. 2025LLM in the Middle: A Systematic Review of Threats and Mitigations to Real-World LLM-based Systemscited by 1
  33. 2025Multi-Agent Framework for Threat Mitigation and Resilience in AI-Based Systemscited by 1
  34. 2026From Thinker to Society: Security in Hierarchical Autonomy Evolution of AI Agentscited by 1
  35. 2026Securing Agentic AI: From Per-Action Checks to Trajectory Assurancecited by 1
  36. 2026EVOMAL: Self-Poisoning in Self-Evolving Coding Agentscited by 1
  37. 2024Can adversarial attacks by large language models be attributed?cited by 0
  38. 2024A Social Outcomes and Priorities centered (SOP) Framework for AI policycited by 0

50 most cited of 58 All 58 (JSON)