mlsanityData to 7 Oct 2026

Abusing Images and Sounds for Indirect Instruction Injection in Multi-Modal LLMs

arXiv 2307.10490 ↗First posted 19 Jul 2023cs.CR cs.AI cs.CLcited by 117

On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness

Abstract (arXiv)

We demonstrate how images and sounds can be used for indirect prompt and instruction injection in multi-modal LLMs. An attacker generates an adversarial perturbation corresponding to the prompt and blends it into an image or audio recording. When the user asks the (unmodified, benign) model about the perturbed image or audio, the perturbation steers the model to output the attacker-chosen text and/or make the subsequent dialog follow the attacker's instruction. We illustrate this attack with several proof-of-concept examples targeting LLaVa and PandaGPT.

Key facts. Cited by 117 papers in the map (data through 7 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 20 of its 23 references that are in the map. First posted on arXiv 19 Jul 2023.

Builds on

20 of 23 references.

References · 20

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2022Training language models to follow instructions with human feedbackcited by 15,712
  5. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  6. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  7. 2023Visual Instruction Tuningcited by 7,792
  8. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  9. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  10. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,339
  11. 2022LaMDA: Language Models for Dialog Applicationscited by 1,218
  12. 2023ImageBind: One Embedding Space To Bind Them Allcited by 1,162
8 more
  1. 2021Transformers in Vision: A Surveycited by 1,009
  2. 2022Mind the Gap: Understanding the Modality Gap in Multi-modal Contrastive Representation Learningcited by 553
  3. 2023PandaGPT: One Model To Instruction-Follow Them Allcited by 330
  4. 2023Are aligned neural networks adversarially aligned?cited by 305
  5. 2018Adversarial Example Generation with Syntactically Controlled Paraphrase Networkscited by 282
  6. 2017Generating Natural Adversarial Examplescited by 182
  7. 2018On Adversarial Examples for Character-Level Neural Machine Translationcited by 85
  8. 2020Adversarial Semantic Collisionscited by 42

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 11; 2024: 36; 2025: 35; 2026: 35.

117 papers.

Sort:

By area:

  1. 2023Jailbreak in pieces: Compositional Adversarial Attacks on Multi-Modal Language Modelscited by 260
  2. 2023Survey of Vulnerabilities in Large Language Models Revealed by Adversarial Attackscited by 189
  3. 2023Image Hijacks: Adversarial Images can Control Generative Models at Runtimecited by 165
  4. 2023On the Adversarial Robustness of Multi-Modal Foundation Modelscited by 142
  5. 2024The Emerged Security and Privacy of LLM Agent: A Survey with Case Studiescited by 127
  6. 2024Eyes Closed, Safety On: Protecting Multimodal LLMs via Image-to-Text Transformationcited by 114
  7. 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 110
  8. 2025A Survey on Trustworthy LLM Agents: Threats and Countermeasurescited by 93
  9. 2024Coercing LLMs to do and reveal (almost) anythingcited by 88
  10. 2024A Survey of Attacks on Large Vision-Language Models: Resources, Advances, and Future Trendscited by 82
  11. 2023Use of LLMs for Illicit Purposes: Threats, Prevention Measures, and Vulnerabilitiescited by 76
  12. 2024Safety of Multimodal Large Language Models on Images and Textscited by 70
38 more
  1. 2024Inducing High Energy-Latency of Large Vision-Language Models with Verbose Imagescited by 69
  2. 2024Imprompter: Tricking LLM Agents into Improper Tool Usecited by 66
  3. 2025A Survey of Safety on Large Vision-Language Models: Attacks, Defenses and Evaluationscited by 66
  4. 2023Ignore This Title and HackAPrompt: Exposing Systemic Vulnerabilities of LLMs through a Global Scale Prompt Hacking Competitioncited by 65
  5. 2025Safety at Scale: A Comprehensive Survey of Large Model and Agent Safetycited by 63
  6. 2024White-box Multimodal Jailbreaks Against Large Vision-Language Modelscited by 57
  7. 2024SelfDefend: LLMs Can Defend Themselves against Jailbreaking in a Practical Mannercited by 57
  8. 2025From Prompt Injections to Protocol Exploits: Threats in LLM-Powered AI Agents Workflowscited by 57
  9. 2024LLMmap: Fingerprinting For Large Language Modelscited by 56
  10. 2023LLM Platform Security: Applying a Systematic Evaluation Framework to OpenAI's ChatGPT Pluginscited by 49
  11. 2024Steering Away from Harm: An Adaptive Approach to Defending Vision Language Model Against Jailbreakscited by 44
  12. 2025Manipulating Multimodal Agents via Cross-Modal Prompt Injectioncited by 42
  13. 2023Misusing Tools in Large Language Models With Visual Adversarial Examplescited by 40
  14. 2025Visual Large Language Models for Generalized and Specialized Applicationscited by 35
  15. 2024IDEATOR: Jailbreaking and Benchmarking Large Vision-Language Models Using Themselvescited by 34
  16. 2025Towards Robust and Secure Embodied AI: A Survey on Vulnerabilities and Attackscited by 32
  17. 2023Open-Sourcing Highly Capable Foundation Models: An evaluation of risks, benefits, and alternative methods for pursuing open-source objectivescited by 31
  18. 2026Prompt Injection Attacks on Agentic Coding Assistants: A Systematic Analysis of Vulnerabilities in Skills, Tools, and Protocol Ecosystemscited by 30
  19. 2025HiddenDetect: Detecting Jailbreak Attacks against Large Vision-Language Models via Monitoring Hidden Statescited by 29
  20. 2023Adversarial Illusions in Multi-Modal Embeddingscited by 28
  21. 2024BlueSuffix: Reinforced Blue Teaming for Vision-Language Models Against Jailbreak Attackscited by 28
  22. 2025Agentic AI Security: Threats, Defenses, Evaluation, and Open Challengescited by 26
  23. 2024Adversarial Robustness for Visual Grounding of Multimodal Large Language Modelscited by 23
  24. 2024A Trembling House of Cards? Mapping Adversarial Attacks against Language Agentscited by 22
  25. 2024Adversarial Attacks of Vision Tasks in the Past 10 Years: A Surveycited by 22
  26. 2024The Wolf Within: Covert Injection of Malice into MLLM Societies via an MLLM Operativecited by 21
  27. 2024Unbridled Icarus: A Survey of the Potential Perils of Image Inputs in Multimodal Large Language Model Securitycited by 18
  28. 2024SpeechGuard: Exploring the Adversarial Robustness of Multimodal Large Language Modelscited by 16
  29. 2025AudioJailbreak: Jailbreak Attacks against End-to-End Large Audio-Language Modelscited by 16
  30. 2024On Catastrophic Inheritance of Large Foundation Modelscited by 15
  31. 2025FC-Attack: Jailbreaking Multimodal Large Language Models via Auto-Generated Flowchartscited by 15
  32. 2025One Surrogate to Fool Them All: Universal, Transferable, and Targeted Adversarial Attacks with CLIPcited by 15
  33. 2024FMM-Attack: A Flow-based Multi-modal Adversarial Attack on Video-based LLMscited by 14
  34. 2024Energy-Latency Manipulation of Multi-modal Large Language Models via Verbose Samplescited by 14
  35. 2024The Synergy between Data and Multi-Modal Large Language Models: A Survey from Co-Development Perspectivecited by 14
  36. 2024Rag and Roll: An End-to-End Evaluation of Indirect Prompt Manipulations in LLM-based Application Frameworkscited by 14
  37. 2025Survey of Adversarial Robustness in Multimodal Large Language Modelscited by 14
  38. 2026Ghost in the Agent: Redefining Information Flow Tracking for LLM Agentscited by 14

50 most cited of 117 All 117 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025ARGUS: Defending Against Multimodal Indirect Prompt Injection via Steering Instruction-Following Behavior
  2. 2026Render Before Reading: Visual Rendering as a Prompt Injection Defense
  3. 2025Investigating Vulnerabilities and Defenses Against Audio-Visual Attacks: A Comprehensive Survey Emphasizing Multimodal Models
  4. 2026A Cross-Modal Prompt Injection Attack against Large Vision-Language Models with Image-Only Perturbation
  5. 2025Beyond Text: Multimodal Jailbreaking of Vision-Language and Audio Models through Perceptually Simple Transformations
  6. 2025Multilingual and Multi-Accent Jailbreaking of Audio LLMs
  7. 2025Breaking Audio Large Language Models by Attacking Only the Encoder: A Universal Targeted Latent-Space Audio Attack
  8. 2026Covert Visual Prompt Injection against Commercial Multimodal Large Language Models