# Adversarial Evaluation of Multimodal Models under Realistic Gray Box Assumption

arXiv: https://arxiv.org/abs/2011.12902 · HTML: https://mlsanity.com/paper/2011.12902/

## Key facts

- Cited by 8 papers in the map (data through 7 Oct 2026).
- First posted on arXiv 25 Nov 2020.

## Built on by (most cited first, 8 of 8)

- [On the Adversarial Robustness of Multi-Modal Foundation Models](https://mlsanity.com/paper/2308.10741/) (2023)
- [Toxic Memes: A Survey of Computational Perspectives on the Detection and Explanation of Meme Toxicities](https://mlsanity.com/paper/2406.07353/) (2024)
- [HateProof: Are Hateful Meme Detection Systems really Robust?](https://mlsanity.com/paper/2302.05703/) (2023)
- [Cross-Modal Attribute Insertions for Assessing the Robustness of Vision-and-Language Learning](https://mlsanity.com/paper/2306.11065/) (2023)
- [URET: Universal Robustness Evaluation Toolkit (for Evasion)](https://mlsanity.com/paper/2308.01840/) (2023)
- [Unaligning Everything: Or Aligning Any Text to Any Image in Multimodal Models](https://mlsanity.com/paper/2407.01157/) (2024)
- [AI Safety in Practice: Enhancing Adversarial Robustness in Multimodal Image Captioning](https://mlsanity.com/paper/2407.21174/) (2024)
- [Black-box Attacks on Image Activity Prediction and its Natural Language Explanations](https://mlsanity.com/paper/2310.00503/) (2023)

## Abstract (arXiv)

This work examines the vulnerability of multimodal (image + text) models to adversarial threats similar to those discussed in previous literature on unimodal (image- or text-only) models. We introduce realistic assumptions of partial model knowledge and access, and discuss how these assumptions differ from the standard "black-box"/"white-box" dichotomy common in current literature on adversarial attacks. Working under various levels of these "gray-box" assumptions, we develop new attack methodologies unique to multimodal classification and evaluate them on the Hateful Memes Challenge classification task. We find that attacking multiple modalities yields stronger attacks than unimodal attacks alone (inducing errors in up to 73% of cases), and that the unimodal image attacks on multimodal classifiers we explored were stronger than character-based text augmentation attacks (inducing errors on average in 45% and 30% of cases, respectively).