A Two-Stage Globally-Diverse Adversarial Attack for Vision-Language Pre-training Models
On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness
Abstract (arXiv)
Vision-language pre-training (VLP) models are vulnerable to adversarial examples, particularly in black-box scenarios. Existing multimodal attacks often suffer from limited perturbation diversity and unstable multi-stage pipelines. To address these challenges, we propose 2S-GDA, a two-stage globally-diverse attack framework. The proposed method first introduces textual perturbations through a globally-diverse strategy by combining candidate text expansion with globally-aware replacement. To enhance visual diversity, image-level perturbations are generated using multi-scale resizing and block-shuffle rotation. Extensive experiments on VLP models demonstrate that 2S-GDA consistently improves attack success rates over state-of-the-art methods, with gains of up to 11.17% in black-box settings. Our framework is modular and can be easily combined with existing methods to further enhance adversarial transferability.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 15 of its 18 references that are in the map. First posted on arXiv 18 Jan 2026.
Builds on
15 of 18 references.
Closest ancestors · 5
Same direction, newest first
- 2024Semantic-Aligned Adversarial Evolution Triangle for High-Transferability Vision-Language Attack4 Nov
- 2024Boosting Transferability in Vision-Language Attacks via Diversification along the Intersection Region of Adversarial Trajectory19 Mar
- 2024An Image Is Worth 1000 Lies: Adversarial Transferability across Prompts on Vision-Language Models14 Mar
- 2023Set-level Guidance Attack: Boosting Adversarial Transferability of Vision-Language Pre-training Models26 Jul
- 2022Towards Adversarial Attack on Vision-Language Pre-training Models19 Jun
Other references · 10
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
- 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,619
- 2015Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Modelscited by 1,362
- 2020BERT-ATTACK: Adversarial Attack Against BERT Using BERTcited by 491
- 2022Vision-Language Pre-Training with Triple Contrastive Learningcited by 212
- 2023Boosting Adversarial Transferability by Block Shuffle and Rotationcited by 64
- 2021Exploiting BERT For Multimodal Target Sentiment Classification Through Input Space Translationcited by 41
- 2023Improving Visual Quality and Transferability of Adversarial Attacks on Face Recognition Simultaneously with Adversarial Restorationcited by 5
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Grounding-Driven Attack: Improving Encoder-based Adversarial Transferability against Large Vision-Language Models
- 2025Transferable Adversarial Attacks on Black-Box Vision-Language Models
- 2024Probing the Robustness of Vision-Language Pretrained Models: A Multimodal Adversarial Attack Approach
- 2024Efficient and Effective Universal Adversarial Attack against Vision-Language Pre-training Models
- 2026Breaking the weakest link to evade vision language models
- 2026On Success and Simplicity: A Second Look at Transferable Vision-Language Attack Pipeline
- 2024Universal Adversarial Perturbations for Vision-Language Pre-trained Models
- 2023Exploring Transferability of Multimodal Adversarial Samples for Vision-Language Pre-training Models with Contrastive Learning