mlsanityData to 14 Oct 2026

On the Feasibility of Poisoning Text-to-Image AI Models via Adversarial Mislabeling

arXiv 2506.21874 ↗First posted 27 Jun 2025cs.CR cs.AIcited by 1

On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness

Abstract (arXiv)

Today's text-to-image generative models are trained on millions of images sourced from the Internet, each paired with a detailed caption produced by Vision-Language Models (VLMs). This part of the training pipeline is critical for supplying the models with large volumes of high-quality image-caption pairs during training. However, recent work suggests that VLMs are vulnerable to stealthy adversarial attacks, where adversarial perturbations are added to images to mislead the VLMs into producing incorrect captions. In this paper, we explore the feasibility of adversarial mislabeling attacks on VLMs as a mechanism to poisoning training pipelines for text-to-image models. Our experiments demonstrate that VLMs are highly vulnerable to adversarial perturbations, allowing attackers to produce benign-looking images that are consistently miscaptioned by the VLM models. This has the effect of injecting strong "dirty-label" poison samples into the training pipeline for text-to-image models, successfully altering their behavior with a small number of poisoned samples. We find that while potential defenses can be effective, they can be targeted and circumvented by adaptive attackers. This suggests a cat-and-mouse game that is likely to reduce the quality of training data and increase the cost of text-to-image model development. Finally, we demonstrate the real-world effectiveness of these attacks, achieving high attack success (over 73%) even in black-box scenarios against commercial VLMs (Google Vertex AI and Microsoft Azure).

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 52 of its 82 references that are in the map. First posted on arXiv 27 Jun 2025.

Builds on

52 of 82 references.

Closest ancestors · 6

Same direction, newest first

  1. 2024Visual Adversarial Attack on Vision-Language Models for Autonomous Driving27 Nov
  2. 2024Revisiting the Adversarial Robustness of Vision Language Models: a Multimodal Perspective30 Apr
  3. 2024Shadowcast: Stealthy Data Poisoning Attacks Against Vision-Language Models5 Feb
  4. 2024Inducing High Energy-Latency of Large Vision-Language Models with Verbose Images20 Jan
  5. 2023How Robust is Google's Bard to Adversarial Image Attacks?21 Sep
  6. 2023Jailbreak in pieces: Compositional Adversarial Attacks on Multi-Modal Language Models26 Jul

Other references · 46

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Denoising Diffusion Implicit Modelscited by 9,781
  3. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  4. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  5. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  6. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
  7. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
  8. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
  9. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
  10. 2021Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Conceptscited by 1,216
  11. 2022Reproducible scaling laws for contrastive language-image learningcited by 1,173
  12. 2017Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoningcited by 1,055
34 more
  1. 2022Exploring CLIP for Assessing the Look and Feel of Imagescited by 991
  2. 2020On Adaptive Attacks to Adversarial Example Defensescited by 555
  3. 2017Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methodscited by 546
  4. 2016Delving into Transferable Adversarial Examples and Black-box Attackscited by 536
  5. 2018Texygen: A Benchmarking Platform for Text Generation Modelscited by 491
  6. 2024SANA: Efficient High-Resolution Image Synthesis with Linear Diffusion Transformerscited by 340
  7. 2023Are aligned neural networks adversarially aligned?cited by 315
  8. 2024Hunyuan-DiT: A Powerful Multi-Resolution Diffusion Transformer with Fine-Grained Chinese Understandingcited by 298
  9. 2023Poisoning Web-Scale Training Datasets is Practicalcited by 290
  10. 2023Driving with LLMs: Fusing Object-Level Vector Modality for Explainable Autonomous Drivingcited by 232
  11. 2020BadNL: Backdoor Attacks against NLP Models with Semantic-preserving Improvementscited by 211
  12. 2016A study of the effect of JPG compression on adversarial imagescited by 197
  13. 2023Image Hijacks: Adversarial Images can Control Generative Models at Runtimecited by 172
  14. 2020Backdoor Attacks Against Deep Learning Systems in the Physical Worldcited by 134
  15. 2023Reason2Drive: Towards Interpretable and Chain-based Reasoning for Autonomous Drivingcited by 123
  16. 2024Dissecting Adversarial Robustness of Multimodal LM Agentscited by 120
  17. 2024A Note on Shumailov et al. (2024): `AI Models Collapse When Trained on Recursively Generated Data'cited by 115
  18. 2023Improving Multimodal Datasets with Image Captioningcited by 93
  19. 2023Text-to-Image Diffusion Models can be Easily Backdoored through Multimodal Data Poisoningcited by 92
  20. 2023Rethinking Model Ensemble in Transfer-based Adversarial Attackscited by 89
  21. 2023TrojDiff: Trojan Attacks on Diffusion Models with Diverse Targetscited by 88
  22. 2023Nightshade: Prompt-Specific Poisoning Attacks on Text-to-Image Generative Modelscited by 75
  23. 2020Bullseye Polytope: A Scalable Clean-Label Poisoning Attack with Improved Transferabilitycited by 66
  24. 2019Stateful Detection of Black-Box Adversarial Attackscited by 56
  25. 2019Enhancing Adversarial Defense by k-Winners-Take-Allcited by 51
  26. 2019A New Defense Against Adversarial Images: Turning a Weakness into a Strengthcited by 49
  27. 2023A Picture is Worth a Thousand Words: Principled Recaptioning Improves Image Generationcited by 41
  28. 2020Blacklight: Scalable Defense for Neural Networks against Query-Based Black-Box Attackscited by 37
  29. 2023Adversarial Illusions in Multi-Modal Embeddingscited by 29
  30. 2018Data Poisoning Attacks against Online Learningcited by 27
  31. 2021Automated Discovery of Adaptive Attacks on Adversarial Defensescited by 22
  32. 2023Differentiable JPEG: The Devil is in the Detailscited by 13
  33. 2016Assessing Threat of Adversarial Examples on Deep Neural Networkscited by 9
  34. 2024Understanding Implosion in Text-to-Image Generative Modelscited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026MemCatalyst: Amplifying Data Auditing on Vision-Language Models via Data Poisoningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Transferable Adversarial Attacks on Black-Box Vision-Language Models
  2. 2025Attention! Your Vision Language Model Could Be Maliciously Manipulated
  3. 2024White-box Multimodal Jailbreaks Against Large Vision-Language Models
  4. 2024Break the Visual Perception: Adversarial Attacks Targeting Encoded Visual Tokens of Large Vision-Language Models
  5. 2025An Image Is Worth Ten Thousand Words: Verbose-Text Induction Attacks on VLMs
  6. 2026Breaking the weakest link to evade vision language models
  7. 2025JaiLIP: Jailbreaking Vision-Language Models via Loss Guided Image Perturbation
  8. 2025Adversarial Confusion Attack: Disrupting Multimodal Large Language Models