mlsanityData to 7 Oct 2026

On Adversarial Vulnerability of Vision-Language Models through the Lens of Intermediate Spectral Subspaces

arXiv 2607.07375 ↗First posted 8 Jul 2026cs.LG cs.AIcited by 0

On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness

Abstract (arXiv)

Adversarial vulnerability in deep neural networks (DNNs) has been studied from the perspectives of decision-boundary geometry, feature robustness, input-output Jacobians, and the instability of inverse problems. Here, we focus on the spectral structure of intermediate linear transformations that propagate information through modern DNNs, an unexplored mechanism of adversarial vulnerability. Specifically, we investigate transformer-based vision-language models, whose linear layers admit interpretable spectral decompositions and whose widespread adoption makes understanding their robustness increasingly important. We propose a white-box spectral-subspace-guided attack (SSGRA) that aligns intermediate representations with the subspace spanned by the bottom right singular vectors. Our experiments show improved attack effectiveness over existing baselines. In addition, SSGRA offers a spectral interpretation of adversarial vulnerability in VLMs, providing insights for improving their robustness.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 49 of its 64 references that are in the map. First posted on arXiv 8 Jul 2026.

Builds on

49 of 64 references.

Closest ancestors · 5

Same direction, newest first

  1. 2024Adversarial Robustness for Visual Grounding of Multimodal Large Language Models16 May
  2. 2024An Image Is Worth 1000 Lies: Adversarial Transferability across Prompts on Vision-Language Models14 Mar
  3. 2024Inducing High Energy-Latency of Large Vision-Language Models with Verbose Images20 Jan
  4. 2023On the Robustness of Large Multimodal Models Against Image Adversarial Attacks6 Dec
  5. 2023How Robust is Google's Bard to Adversarial Image Attacks?21 Sep

Other references · 44

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  4. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  5. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  6. 2013Intriguing properties of neural networkscited by 5,114
  7. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,976
  8. 2017Improved Training of Wasserstein GANscited by 2,223
  9. 2018Spectral Normalization for Generative Adversarial Networkscited by 1,618
  10. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,510
  11. 2019Adversarial Examples Are Not Bugs, They Are Featurescited by 950
  12. 2016Universal adversarial perturbationscited by 774
32 more
  1. 2018Lipschitz regularity of deep neural networks: analysis and efficient estimationcited by 413
  2. 2018Defense-GAN: Protecting Classifiers Against Adversarial Attacks Using Generative Modelscited by 362
  3. 2019Efficient and Accurate Estimation of Lipschitz Constants for Deep Neural Networkscited by 290
  4. 2017Parseval Networks: Improving Robustness to Adversarial Examplescited by 288
  5. 2017PixelDefend: Leveraging Generative Models to Understand and Defend against Adversarial Examplescited by 243
  6. 2025LLaVA-OneVision-1.5: Fully Open Framework for Democratized Multimodal Trainingcited by 233
  7. 2018Implicit Self-Regularization in Deep Neural Networks: Evidence from Random Matrix Theory and Implications for Learningcited by 211
  8. 2019On instabilities of deep learning in image reconstruction - Does AI come at a cost?cited by 211
  9. 2017Formal Guarantees on the Robustness of a Classifier against Adversarial Manipulationcited by 182
  10. 2020A Self-supervised Approach for Adversarial Robustnesscited by 180
  11. 2017Spectral Norm Regularization for Improving the Generalizability of Deep Learningcited by 178
  12. 2023Image Hijacks: Adversarial Images can Control Generative Models at Runtimecited by 165
  13. 2019Robust Learning with Jacobian Regularizationcited by 127
  14. 2018Improving DNN Robustness to Adversarial Attacks using Jacobian Regularizationcited by 88
  15. 2019On the Connection Between Adversarial Robustness and Saliency Map Interpretabilitycited by 80
  16. 2015Analysis of classifiers' robustness to adversarial perturbationscited by 76
  17. 2023VLATTACK: Multimodal Adversarial Attacks on Vision-Language Tasks via Pre-trained Modelscited by 66
  18. 2015Adversarial Manipulation of Deep Representationscited by 56
  19. 2019FDA: Feature Disruptive Attackcited by 55
  20. 2017Art of singular vectors and universal adversarial perturbationscited by 49
  21. 2019Enhancing Cross-task Black-Box Transferability of Adversarial Examples with Dispersion Reductioncited by 47
  22. 2018On the Geometry of Adversarial Examplescited by 43
  23. 2025Adversarial Attacks against Closed-Source MLLMs via Feature Optimal Alignmentcited by 42
  24. 2019Jacobian Adversarially Regularized Networks for Robustnesscited by 40
  25. 2020The troublesome kernel -- On hallucinations, no free lunches and the accuracy-stability trade-off in inverse problemscited by 40
  26. 2015Foveation-based Mechanisms Alleviate Adversarial Examplescited by 32
  27. 2024Stop Reasoning! When Multimodal LLM with Chain-of-Thought Reasoning Meets Adversarial Imagecited by 26
  28. 2018Adversarial Attacks on Variational Autoencoderscited by 19
  29. 2021Certifiably Robust Variational Autoencoderscited by 14
  30. 2020Improved Gradient based Adversarial Attacks for Quantized Networkscited by 10
  31. 2021Diagnosing Vulnerability of Variational Auto-Encoders to Adversarial Attackscited by 7
  32. 2023Ensuring Topological Data-Structure Preservation under Autoencoder Compression due to Latent Space Regularization in Gauss--Legendre nodescited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025High-Entropy Tokens as Multimodal Failure Points in Vision-Language Models
  2. 2025EigenShield: Causal Subspace Filtering via Random Matrix Theory for Adversarially Robust Vision-Language Models
  3. 2026What Makes VLMs Robust? Towards Reconciling Robustness and Accuracy in Vision-Language Models
  4. 2025Navigating the Trade-off: A Synthesis of Defensive Strategies for Zero-Shot Adversarial Robustness in Vision-Language Models
  5. 2024Towards Adversarially Robust Vision-Language Models: Insights from Design Choices and Prompt Formatting Techniques
  6. 2025Transferable Adversarial Attacks on Black-Box Vision-Language Models
  7. 2026Breaking the weakest link to evade vision language models
  8. 2026Adversarial Attacks Already Tell the Answer: Directional Bias-Guided Test-time Defense for Vision-Language Models