mlsanityData to 7 Oct 2026

Text Prompt Injection of Vision Language Models

arXiv 2510.09849 ↗First posted 10 Oct 2025cs.CL cs.CVcited by 1

On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness

also in 1 more
Abstract (arXiv)

The widespread application of large vision language models has significantly raised safety concerns. In this project, we investigate text prompt injection, a simple yet effective method to mislead these models. We developed an algorithm for this type of attack and demonstrated its effectiveness and efficiency through experiments. Compared to other attack methods, our approach is particularly effective for large models without high demand for computational resources.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 20 of its 26 references that are in the map. First posted on arXiv 10 Oct 2025.

Builds on

20 of 26 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024White-box Multimodal Jailbreaks Against Large Vision-Language Models28 May
  2. 2024Test-Time Backdoor Attacks on Multimodal Large Language Models13 Feb
  3. 2023How Robust is Google's Bard to Adversarial Image Attacks?21 Sep
  4. 2023On Evaluating Adversarial Robustness of Large Vision-Language Models26 May

Other references · 16

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  4. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  5. 2013Intriguing properties of neural networkscited by 5,114
  6. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  7. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  8. 2015DeepFool: a simple and accurate method to fool deep neural networkscited by 1,315
  9. 2015The Limitations of Deep Learning in Adversarial Settingscited by 784
  10. 2024PaliGemma: A versatile 3B VLM for transfercited by 679
  11. 2020RobustBench: a standardized adversarial robustness benchmarkcited by 659
  12. 2023MiniGPT-v2: large language model as a unified interface for vision-language multi-task learningcited by 617
4 more
  1. 2022Diffusion Models for Adversarial Purificationcited by 469
  2. 2023Are aligned neural networks adversarially aligned?cited by 305
  3. 2023Universal Jailbreak Backdoors from Poisoned Human Feedbackcited by 120
  4. 2023Rethinking Model Ensemble in Transfer-based Adversarial Attackscited by 88

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025GEO-Detective: Unveiling Location Privacy Risks in Images with LLM Agentscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024A Survey of Attacks on Large Vision-Language Models: Resources, Advances, and Future Trends
  2. 2026Text is All You Need for Vision-Language Model Jailbreaking
  3. 2025A Survey of Safety on Large Vision-Language Models: Attacks, Defenses and Evaluations
  4. 2026Physical Prompt Injection Attacks on Large Vision-Language Models
  5. 2025Attention! Your Vision Language Model Could Be Maliciously Manipulated
  6. 2026A Cross-Modal Prompt Injection Attack against Large Vision-Language Models with Image-Only Perturbation
  7. 2026Covert Visual Prompt Injection against Commercial Multimodal Large Language Models
  8. 2026On Success and Simplicity: A Second Look at Transferable Vision-Language Attack Pipeline