Laundering AI Authority with Adversarial Examples
On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness
also in 1 more
Abstract (arXiv)
Vision-language models (VLMs) are increasingly deployed as trusted authorities -- fact-checking images on social media, comparing products, and moderating content. Users implicitly trust that these systems perceive the same visual content as they do. We show that adversarial examples break this assumption, enabling AI authority laundering: an attacker subtly perturbs an image so that the VLM produces confident and authoritative responses about the wrong input. Unlike jailbreaks or prompt injections, our attacks do not compromise model alignment; the attack operates entirely at the perceptual level. We demonstrate that standard attacks against publicly available CLIP models transfer reliably to production VLMs -- including GPT-5.4, Claude Opus~4.6, Gemini~3, and Grok~4.2. Across four attack surfaces, we show that authority laundering can amplify misinformation, disparage individuals, evade content moderation, and manipulate product recommendations. Our attacks have high success rates: In hundreds of attacks targeting identity manipulation and NSFW evasion, we measure success rates of 22 - 100% across six models. No novel attack algorithm is required: basic techniques known for over a decade suffice, establishing a lower bound on attacker capability that should concern defenders. Our results demonstrate that visual adversarial robustness is now a practical -- and still largely unsolved -- safety problem.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 32 of its 61 references that are in the map. First posted on arXiv 5 May 2026.
Builds on
32 of 61 references.
Closest ancestors · 6
Same direction, newest first
- 2025A Frustratingly Simple Yet Highly Effective Attack Baseline: Over 90% Success Rate Against the Strong Black-box Models of GPT-4.5/4o/o113 Mar
- 2024Failures to Find Transferable Image Jailbreaks Between Vision-Language Models21 Jul
- 2023On the Robustness of Large Multimodal Models Against Image Adversarial Attacks6 Dec
- 2023Jailbreak in pieces: Compositional Adversarial Attacks on Multi-Modal Language Models26 Jul
- 2023Abusing Images and Sounds for Indirect Instruction Injection in Multi-Modal LLMs19 Jul
- 2023Visual Adversarial Examples Jailbreak Aligned Large Language Models22 Jun
Other references · 26
- 2014Explaining and Harnessing Adversarial Examplescited by 7,287
- 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
- 2013Intriguing properties of neural networkscited by 5,114
- 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
- 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
- 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,510
- 2019Certified Adversarial Robustness via Randomized Smoothingcited by 1,324
- 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,259
- 2022Red Teaming Language Models with Language Modelscited by 983
- 2019Adversarial Examples Are Not Bugs, They Are Featurescited by 950
- 2018Robustness May Be at Odds with Accuracycited by 873
- 2018Improving Transferability of Adversarial Examples with Input Diversitycited by 591
14 more
- 2020On Adaptive Attacks to Adversarial Example Defensescited by 547
- 2017Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methodscited by 542
- 2016Delving into Transferable Adversarial Examples and Black-box Attackscited by 530
- 2018Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Surveycited by 459
- 2016Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samplescited by 439
- 2023Are aligned neural networks adversarially aligned?cited by 305
- 2023Image Hijacks: Adversarial Images can Control Generative Models at Runtimecited by 165
- 2020Randomized Smoothing of All Shapes and Sizescited by 146
- 2021Advances in adversarial attacks and defenses in computer vision: A surveycited by 127
- 2019Towards Large yet Imperceptible Adversarial Image Perturbations with Perceptual Color Distancecited by 72
- 2021Spinning Language Models: Risks of Propaganda-As-A-Service and Countermeasurescited by 69
- 2019AdvFaces: Adversarial Face Synthesiscited by 59
- 2024AI-generated faces influence gender stereotypes and racial homogenizationcited by 13
- 2024Gradient-based Jailbreak Images for Multimodal Fusion Modelscited by 13
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Attention! Your Vision Language Model Could Be Maliciously Manipulated
- 2024Self-interpreting Adversarial Images
- 2026Making MLLMs Blind: Adversarial Smuggling Attacks in MLLM Content Moderation
- 2026Leave My Images Alone: Preventing Multi-Modal Large Language Models from Analyzing Images via Visual Prompt Injection
- 2025Tracking the Copyright of Large Vision-Language Models through Parameter Learning Adversarial Images
- 2025On the Feasibility of Poisoning Text-to-Image AI Models via Adversarial Mislabeling
- 2025VIP: Visual Information Protection through Adversarial Attacks on Vision-Language Models
- 2025Transferable Adversarial Attacks on Black-Box Vision-Language Models