The Security Threat of Compressed Projectors in Large Vision-Language Models
On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness
Abstract (arXiv)
The choice of a suitable visual language projector (VLP) is critical to the successful training of large visual language models (LVLMs). Mainstream VLPs can be broadly categorized into compressed and uncompressed projectors, and each offers distinct advantages in performance and computational efficiency. However, their security implications have not been thoroughly examined. Our comprehensive evaluation reveals significant differences in their security profiles: compressed projectors exhibit substantial vulnerabilities, allowing adversaries to successfully compromise LVLMs even with minimal knowledge of structure information. In stark contrast, uncompressed projectors demonstrate robust security properties and do not introduce additional vulnerabilities. These findings provide critical guidance for researchers in selecting optimal VLPs that enhance the security and reliability of visual language models. The code is available at https://github.com/btzyd/TCP.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 18 of its 31 references that are in the map. First posted on arXiv 31 May 2025.
Builds on
18 of 31 references.
Closest ancestors · 1
Same direction, newest first
Other references · 17
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2014Microsoft COCO: Common Objects in Contextcited by 17,498
- 2014Explaining and Harnessing Adversarial Examplescited by 7,287
- 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
- 2013Intriguing properties of neural networkscited by 5,114
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
- 2016Towards Evaluating the Robustness of Neural Networkscited by 2,976
- 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
- 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
- 2015VQA: Visual Question Answeringcited by 2,489
- 2014CIDEr: Consensus-based Image Description Evaluationcited by 2,027
- 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,326
5 more
- 2014Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Imagescited by 774
- 2022EVA: Exploring the Limits of Masked Visual Representation Learning at Scalecited by 685
- 2018Improving Transferability of Adversarial Examples with Input Diversitycited by 591
- 2023TimeChat: A Time-sensitive Multimodal Large Language Model for Long Video Understandingcited by 399
- 2023Honeybee: Locality-enhanced Projector for Multimodal LLMcited by 195
Built on by
Citing papers per year: 2025: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026Less Is More -- Until It Breaks: Security Pitfalls of Vision Token Compression in Large Vision-Language Models
- 2025A Survey of Safety on Large Vision-Language Models: Attacks, Defenses and Evaluations
- 2025When Data Manipulation Meets Attack Goals: An In-depth Survey of Attacks for VLMs
- 2024Towards Adversarially Robust Vision-Language Models: Insights from Design Choices and Prompt Formatting Techniques
- 2026Mechanistically Interpreting Compression in Vision-Language Models
- 2025Safeguarding Vision-Language Models: Mitigating Vulnerabilities to Gaussian Noise in Perturbation-based Attacks
- 2024A Survey of Attacks on Large Vision-Language Models: Resources, Advances, and Future Trends
- 2026On the Adversarial Robustness of Large Vision-Language Models under Visual Token Compression