mlsanityData to 7 Oct 2026

GeoDetect: Geometric Adversarial Detection for VLPs

arXiv 2607.14737 ↗First posted 16 Jul 2026cs.CV cs.LGcited by 0

On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness

Abstract (arXiv)

Vision-language pre-trained models (VLPs) are widely used in real-world applications. However, they remain vulnerable to adversarial attacks. Although adversarial detection methods have demonstrated success in single-modality settings (either vision or language), their effectiveness and reliability in multimodal models such as VLPs remain largely unexplored. In this work, we study the geometry of VLP embedding spaces and observe structured anisotropy that differs from unimodal vision models. Our theoretical analysis shows that under this anisotropic structure, adversarial attacks increase the expected geometric separation between clean and adversarial examples (AEs). Specifically, we demonstrate that AEs consistently exhibit greater expected distances to randomly sampled points than their clean counterparts, indicating that AEs tend to push representations out of manifold regions. Building on these insights, we propose GeoDetect, which leverages these off-manifold deviations via geometric scores to identify AEs. Through comprehensive evaluations, we show that our approach reliably detects AEs across diverse VLP architectures and threat settings, covering unimodal and multimodal attacks as well as adaptive attacks, thereby providing a robust and practical approach to improving the safety and reliability of these models.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 46 of its 70 references that are in the map. First posted on arXiv 16 Jul 2026.

Builds on

46 of 70 references.

Closest ancestors · 11

Same direction, newest first

  1. 2024PIP: Detecting Adversarial Examples in Large Vision-Language Models via Attention Patterns of Irrelevant Probe Questions8 Sep
  2. 2024Few-Shot Adversarial Prompt Learning on Vision-Language Models21 Mar
  3. 2024One Prompt Word is Enough to Boost Adversarial Robustness for Pre-trained Vision-Language Models4 Mar
  4. 2024Robust CLIP: Unsupervised Adversarial Fine-Tuning of Vision Embeddings for Robust Large Vision-Language Models19 Feb
  5. 2024Pre-trained Model Guided Fine-Tuning for Zero-Shot Adversarial Robustness9 Jan
  6. 2023SA-Attack: Improving Adversarial Transferability of Vision-Language Pre-training Models via Self-Augmentation8 Dec
  7. 2023OT-Attack: Enhancing Adversarial Transferability of Vision-Language Models via Optimal Transport Optimization7 Dec
  8. 2023Adversarial Prompt Tuning for Vision-Language Models19 Nov
  9. 2023Set-level Guidance Attack: Boosting Adversarial Transferability of Vision-Language Pre-training Models26 Jul
  10. 2022Understanding Zero-Shot Adversarial Robustness for Large-Scale Models14 Dec
  11. 2022Towards Adversarial Attack on Vision-Language Pre-training Models19 Jun

Other references · 35

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  3. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  4. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  5. 2013Intriguing properties of neural networkscited by 5,114
  6. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  7. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,219
  8. 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,612
  9. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,510
  10. 2018A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attackscited by 1,472
  11. 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,259
  12. 2023A Survey on Multimodal Large Language Modelscited by 898
23 more
  1. 2018Robustness May Be at Odds with Accuracycited by 873
  2. 2022Mind the Gap: Understanding the Modality Gap in Multi-modal Contrastive Representation Learningcited by 553
  3. 2020BERT-ATTACK: Adversarial Attack Against BERT Using BERTcited by 488
  4. 2022Multimodal Learning with Transformers: A Surveycited by 399
  5. 2017Don't Just Assume; Look and Answer: Overcoming Priors for Visual Question Answeringcited by 337
  6. 2018Characterizing Adversarial Subspaces Using Local Intrinsic Dimensionalitycited by 280
  7. 2017Detecting Adversarial Samples from Artifactscited by 244
  8. 2022Delving into Out-of-Distribution Detection with Vision-Language Representationscited by 215
  9. 2022Vision-Language Pre-Training with Triple Contrastive Learningcited by 212
  10. 2017All-but-the-Top: Simple and Effective Postprocessing for Word Representationscited by 205
  11. 2017On the (Statistical) Detection of Adversarial Examplescited by 180
  12. 2023On the Adversarial Robustness of Multi-Modal Foundation Modelscited by 142
  13. 2020IMRAM: Iterative Matching with Recurrent Attention Memory for Cross-Modal Image-Text Retrievalcited by 115
  14. 2019Cycle-Consistency for Robust Visual Question Answeringcited by 105
  15. 2023AdvCLIP: Downstream-agnostic Adversarial Examples in Multimodal Contrastive Learningcited by 101
  16. 2019Detecting Adversarial Samples Using Influence Functions and Nearest Neighborscited by 72
  17. 2023VLATTACK: Multimodal Adversarial Attacks on Vision-Language Tasks via Pre-trained Modelscited by 66
  18. 2024The Double-Ellipsoid Geometry of CLIPcited by 42
  19. 2021Evading Adversarial Example Detection Defenses with Orthogonal Projected Gradient Descentcited by 34
  20. 2024Revisiting Adversarial Training at Scalecited by 33
  21. 2019Deep Neural Rejection against Adversarial Examplescited by 31
  22. 2024Goal-guided Generative Prompt Injection Attack on Large Language Modelscited by 22
  23. 2021Revisiting Model's Uncertainty and Confidences for Adversarial Example Detectioncited by 9

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026On the Adversarial Robustness of 3D Large Vision-Language Models
  2. 2024Effective and Efficient Adversarial Detection for Vision-Language Models via A Single Vector
  3. 2026Adversarial Attacks Already Tell the Answer: Directional Bias-Guided Test-time Defense for Vision-Language Models
  4. 2026Detecting Adversarial Images through Response Profiles of Vision-Language Models
  5. 2025Robust Vision-Language Models via Tensor Decomposition: A Defense Against Adversarial Attacks
  6. 2026The Geometry of Robustness: Optimizing Loss Landscape Curvature and Feature Manifold Alignment for Robust Finetuning of Vision-Language Models
  7. 2024Multimodal Adversarial Defense for Vision-Language Models by Leveraging One-To-Many Relationships
  8. 2026VALD: Multi-Stage Vision Attack Detection for Efficient LVLM Defense