mlsanityData to 7 Oct 2026

Understanding Adversarial Transfer: Why Representation-Space Attacks Fail Where Data-Space Attacks Succeed

arXiv 2510.01494 ↗First posted 1 Oct 2025cs.LG cs.AIcited by 0

On the map: LLM safety and security → Vision-language and text-to-image safety

Abstract (arXiv)

The field of adversarial robustness has long established that adversarial examples can successfully transfer between image classifiers and that text jailbreaks can successfully transfer between language models (LMs). However, a pair of recent studies reported being unable to successfully transfer image jailbreaks between vision-language models (VLMs). To explain this striking difference, we propose a fundamental distinction regarding the transferability of attacks against machine learning models: attacks in the input data-space can transfer, whereas attacks in model representation space do not, at least not without geometric alignment of representations. We then provide theoretical and empirical evidence of this hypothesis in four different settings. First, we mathematically prove this distinction in a simple setting where two networks compute the same input-output map but via different representations. Second, we construct representation-space attacks against image classifiers that are as successful as well-known data-space attacks, but fail to transfer. Third, we construct representation-space attacks against LMs that successfully jailbreak the attacked models but again fail to transfer. Fourth, we construct data-space attacks against VLMs that successfully transfer to new VLMs, and we show that representation space attacks can transfer when VLMs' latent geometries are sufficiently aligned in post-projector space. Our work reveals that adversarial transfer is not an inherent property of all attacks but contingent on their operational domain - the shared data-space versus models' unique representation spaces - a critical insight for building more robust models.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Vision-language and text-to-image safety, one of 12 directions in LLM safety and security. Builds on 46 of its 69 references that are in the map. First posted on arXiv 1 Oct 2025.

Builds on

46 of 69 references.

Closest ancestors · 8

Same direction, newest first

  1. 2024IDEATOR: Jailbreaking and Benchmarking Large Vision-Language Models Using Themselves29 Oct
  2. 2024Gradient-based Jailbreak Images for Multimodal Fusion Models4 Oct
  3. 2024Failures to Find Transferable Image Jailbreaks Between Vision-Language Models21 Jul
  4. 2024Images are Achilles' Heel of Alignment: Exploiting Visual Vulnerabilities for Jailbreaking Multimodal Large Language Models14 Mar
  5. 2024Safety Fine-Tuning at (Almost) No Cost: A Baseline for Vision Large Language Models3 Feb
  6. 2023FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Prompts9 Nov
  7. 2023Jailbreak in pieces: Compositional Adversarial Attacks on Multi-Modal Language Models26 Jul
  8. 2023Visual Adversarial Examples Jailbreak Aligned Large Language Models22 Jun

Other references · 38

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  4. 2023Visual Instruction Tuningcited by 7,792
  5. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  6. 2013Intriguing properties of neural networkscited by 5,114
  7. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  8. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  9. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  10. 2019Similarity of Neural Network Representations Revisitedcited by 1,695
  11. 2016Concrete Problems in AI Safetycited by 1,582
  12. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,326
26 more
  1. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,106
  2. 2023A Survey on Multimodal Large Language Modelscited by 898
  3. 2016Practical Black-Box Attacks against Machine Learningcited by 893
  4. 2016Universal adversarial perturbationscited by 774
  5. 2023Tree of Attacks: Jailbreaking Black-Box LLMs Automaticallycited by 619
  6. 2016Delving into Transferable Adversarial Examples and Black-box Attackscited by 530
  7. 2023DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Modelscited by 505
  8. 2024Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attackscited by 450
  9. 2016Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samplescited by 439
  10. 2024The Platonic Representation Hypothesiscited by 362
  11. 2024Prismatic VLMs: Investigating the Design Space of Visually-Conditioned Language Modelscited by 336
  12. 2023On Evaluating Adversarial Robustness of Large Vision-Language Modelscited by 281
  13. 2021Revisiting Model Stitching to Compare Neural Representationscited by 203
  14. 2025Constitutional Classifiers: Defending against Universal Jailbreaks across Thousands of Hours of Red Teamingcited by 172
  15. 2017The Space of Transferable Adversarial Examplescited by 156
  16. 2023Abusing Images and Sounds for Indirect Instruction Injection in Multi-Modal LLMscited by 117
  17. 2025Which Economic Tasks are Performed with AI? Evidence from Millions of Claude Conversationscited by 88
  18. 2025Design Patterns for Securing LLM Agents against Prompt Injectionscited by 67
  19. 2025Artificial Intelligence Index Report 2025cited by 58
  20. 2025Harnessing the Universal Geometry of Embeddingscited by 54
  21. 2021Closer Look at the Transferability of Adversarial Examples: How They Fool Different Models Differentlycited by 22
  22. 2020Uncovering the Connections Between Adversarial Transferability and Knowledge Transferabilitycited by 20
  23. 2023Evading Black-box Classifiers Without Breaking Eggscited by 13
  24. 2025Independence Tests for Language Modelscited by 10
  25. 2025Transferable Adversarial Attacks on Black-Box Vision-Language Modelscited by 10
  26. 2021Disrupting Adversarial Transferability in Deep Neural Networkscited by 6

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023OT-Attack: Enhancing Adversarial Transferability of Vision-Language Models via Optimal Transport Optimization
  2. 2026One Attack to Fool Them All: Highly Transferable Black-Box Adversarial Attacks on Frontier MLLMs
  3. 2024Transfer Attack for Bad and Good: Explain and Boost Adversarial Transferability across Multimodal Large Language Models
  4. 2025FORCE: Transferable Visual Jailbreaking Attacks via Feature Over-Reliance CorrEction
  5. 2023Rethinking Model Ensemble in Transfer-based Adversarial Attacks
  6. 2024Chain of Attack: On the Robustness of Vision-Language Models Against Transfer-Based Adversarial Attacks
  7. 2026Latent Transfer Attack: Adversarial Examples via Generative Latent Spaces
  8. 2023SA-Attack: Improving Adversarial Transferability of Vision-Language Pre-training Models via Self-Augmentation