mlsanityData to 7 Oct 2026

As Firm As Their Foundations: Can open-sourced foundation models be used to create adversarial examples for downstream tasks?

arXiv 2403.12693 ↗First posted 19 Mar 2024cs.CVcited by 11

Not on the map. Why?

Abstract (arXiv)

Foundation models pre-trained on web-scale vision-language data, such as CLIP, are widely used as cornerstones of powerful machine learning systems. While pre-training offers clear advantages for downstream learning, it also endows downstream models with shared adversarial vulnerabilities that can be easily identified through the open-sourced foundation model. In this work, we expose such vulnerabilities in CLIP's downstream models and show that foundation models can serve as a basis for attacking their downstream systems. In particular, we propose a simple yet effective adversarial attack strategy termed Patch Representation Misalignment (PRM). Solely based on open-sourced CLIP vision encoders, this method produces adversaries that simultaneously fool more than 20 downstream models spanning 4 common vision-language tasks (semantic segmentation, object detection, image captioning and visual question-answering). Our findings highlight the concerning safety risks introduced by the extensive usage of public foundational models in the development of downstream systems, calling for extra caution in these scenarios.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 97 of its 115 references that are in the map. First posted on arXiv 19 Mar 2024.

Builds on

97 of 115 references.

References · 97

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  5. 2023Visual Instruction Tuningcited by 7,792
  6. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  7. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  8. 2013Intriguing properties of neural networkscited by 5,114
  9. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  10. 2022A ConvNet for the 2020scited by 3,545
  11. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  12. 2016Perceptual Losses for Real-Time Style Transfer and Super-Resolutioncited by 2,678
38 more
  1. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,498
  2. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,219
  3. 2014CIDEr: Consensus-based Image Description Evaluationcited by 2,027
  4. 2019LXMERT: Learning Cross-Modality Encoder Representations from Transformerscited by 1,548
  5. 2015Microsoft COCO Captions: Data Collection and Evaluation Servercited by 1,447
  6. 2019VisualBERT: A Simple and Performant Baseline for Vision and Languagecited by 1,291
  7. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,106
  8. 2019Adversarial Examples Are Not Bugs, They Are Featurescited by 950
  9. 2016COCO-Stuff: Thing and Stuff Classes in Contextcited by 858
  10. 2022Detecting Twenty-thousand Classes using Image-level Supervisioncited by 613
  11. 2018Improving Transferability of Adversarial Examples with Input Diversitycited by 591
  12. 2023OpenFlamingo: An Open-Source Framework for Training Large Autoregressive Vision-Language Modelscited by 560
  13. 2021ClipCap: CLIP Prefix for Image Captioningcited by 557
  14. 2016Delving into Transferable Adversarial Examples and Black-box Attackscited by 530
  15. 2019Evading Defenses to Transferable Adversarial Examples by Translation-Invariant Attackscited by 440
  16. 2016Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samplescited by 439
  17. 2019Nesterov Accelerated Gradient and Scale Invariance for Adversarial Attackscited by 352
  18. 2021How Much Can CLIP Benefit Vision-and-Language Tasks?cited by 321
  19. 2023On Evaluating Adversarial Robustness of Large Vision-Language Modelscited by 281
  20. 2023Poisoning Web-Scale Training Datasets is Practicalcited by 276
  21. 2020Counterfactual VQA: A Cause-Effect Look at Language Biascited by 266
  22. 2023Jailbreak in pieces: Compositional Adversarial Attacks on Multi-Modal Language Modelscited by 260
  23. 2021Enhancing the Transferability of Adversarial Attacks through Variance Tuningcited by 242
  24. 2021A Simple Baseline for Open-Vocabulary Semantic Segmentation with Pre-trained Vision-language Modelcited by 223
  25. 2022Open-Vocabulary DETR with Conditional Matchingcited by 220
  26. 2023CAT-Seg: Cost Aggregation for Open-Vocabulary Semantic Segmentationcited by 215
  27. 2023Convolutions Die Hard: Open-Vocabulary Segmentation with Single Frozen Convolutional CLIPcited by 199
  28. 2023How Robust is Google's Bard to Adversarial Image Attacks?cited by 177
  29. 2022SegCLIP: Patch Aggregation with Learnable Centers for Open-Vocabulary Semantic Segmentationcited by 162
  30. 2017Generative Adversarial Perturbationscited by 153
  31. 2020Counterfactual Samples Synthesizing for Robust Visual Question Answeringcited by 153
  32. 2022Bridging the Gap between Object and Image-level Representations for Open-Vocabulary Detectioncited by 145
  33. 2022Towards Adversarial Attack on Vision-Language Pre-training Modelscited by 142
  34. 2019Enhancing Adversarial Example Transferability with an Intermediate Level Attackcited by 139
  35. 2018Constructing Unrestricted Adversarial Examples with Generative Modelscited by 135
  36. 2023CORA: Adapting CLIP for Open-Vocabulary Detection with Region Prompting and Anchor Pre-Matchingcited by 133
  37. 2022Learning to Generate Text-grounded Mask for Open-world Semantic Segmentation from Only Image-Text Pairscited by 130
  38. 2017On the Robustness of Semantic Segmentation Models to Adversarial Attackscited by 126

50 most cited of 97 All 97 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 5; 2026: 4.

11 papers.

  1. 2024NaturalBench: Evaluating Vision-Language Models on Natural Adversarial Samplescited by 94
  2. 2025Safety at Scale: A Comprehensive Survey of Large Model and Agent Safetycited by 63
  3. 2025X-Transfer Attacks: Towards Super Transferable Adversarial Attacks on CLIPcited by 21
  4. 2024Transfer Attack for Bad and Good: Explain and Boost Adversarial Transferability across Multimodal Large Language Modelscited by 5
  5. 2025VIP: Visual Information Protection through Adversarial Attacks on Vision-Language Modelscited by 4
  6. 2026Contrastive Spectral Rectification: Test-Time Defense towards Zero-shot Adversarial Robustness of CLIPcited by 2
  7. 2025One Object, Multiple Lies: A Benchmark for Cross-task Adversarial Attack on Unified Vision-Language Modelscited by 1
  8. 2026Universal Anti-forensics Attack against Image Forgery Detection via Multi-modal Guidancecited by 1
  9. 2025Quantifying Return on Security Controls in LLM Systemscited by 0
  10. 2026Do Not Leave a Gap: Hallucination-Free Object Concealment in Vision-Language Modelscited by 0
  11. 2026Beyond False Stability: High-Noise Drift Gating for Test-Time Adversarial Defenses in Vision-Language Modelscited by 0