# Adversarial Robustness of Visual Dialog

arXiv: https://arxiv.org/abs/2207.02639 · HTML: https://mlsanity.com/paper/2207.02639/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 37 of its 52 references that are in the map.
- First posted on arXiv 6 Jul 2022.

## Builds on (most cited first, 30 of 37)

- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [VQA: Visual Question Answering](https://mlsanity.com/paper/1505.00468/) (2015)
- [Adversarial examples in the physical world](https://mlsanity.com/paper/1607.02533/) (2016)
- [Adversarial Machine Learning at Scale](https://mlsanity.com/paper/1611.01236/) (2016)
- [Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailment](https://mlsanity.com/paper/1907.11932/) (2019)
- [Improving Transferability of Adversarial Examples with Input Diversity](https://mlsanity.com/paper/1803.06978/) (2018)
- [BERT-ATTACK: Adversarial Attack Against BERT Using BERT](https://mlsanity.com/paper/2004.09984/) (2020)
- [Universal Sentence Encoder](https://mlsanity.com/paper/1803.11175/) (2018)
- [Visual Dialog](https://mlsanity.com/paper/1611.08669/) (2016)
- [Black-box Generation of Adversarial Text Sequences to Evade Deep Learning Classifiers](https://mlsanity.com/paper/1801.04354/) (2018)
- [Deep Modular Co-Attention Networks for Visual Question Answering](https://mlsanity.com/paper/1906.10770/) (2019)
- [Adversarial Example Generation with Syntactically Controlled Paraphrase Networks](https://mlsanity.com/paper/1804.06059/) (2018)
- [Generating Natural Adversarial Examples](https://mlsanity.com/paper/1710.11342/) (2017)
- [Build it Break it Fix it for Dialogue Safety: Robustness from Adversarial Human Attack](https://mlsanity.com/paper/1908.06083/) (2019)
- [Counter-fitting Word Vectors to Linguistic Constraints](https://mlsanity.com/paper/1603.00892/) (2016)
- [Text Processing Like Humans Do: Visually Attacking and Shielding NLP Systems](https://mlsanity.com/paper/1903.11508/) (2019)
- [Reevaluating Adversarial Examples in Natural Language](https://mlsanity.com/paper/2004.14174/) (2020)
- [Two Causal Principles for Improving Visual Dialog](https://mlsanity.com/paper/1911.10496/) (2019)
- [Learning Cooperative Visual Dialog Agents with Deep Reinforcement Learning](https://mlsanity.com/paper/1703.06585/) (2017)
- [Large-scale Pretraining for Visual Dialog: A Simple State-of-the-Art Baseline](https://mlsanity.com/paper/1912.02379/) (2019)
- [Visual Coreference Resolution in Visual Dialog using Neural Module Networks](https://mlsanity.com/paper/1809.01816/) (2018)
- [Do Neural Dialog Systems Use the Conversation History Effectively? An Empirical Study](https://mlsanity.com/paper/1906.01603/) (2019)
- [Simple Black-Box Adversarial Perturbations for Deep Networks](https://mlsanity.com/paper/1612.06299/) (2016)
- [Word-level Textual Adversarial Attacking as Combinatorial Optimization](https://mlsanity.com/paper/1910.12196/) (2019)
- [Recursive Visual Attention in Visual Dialog](https://mlsanity.com/paper/1812.02664/) (2018)
- [History for Visual Dialog: Do we really need it?](https://mlsanity.com/paper/2005.07493/) (2020)
- [Adversarial Over-Sensitivity and Over-Stability Strategies for Dialogue Models](https://mlsanity.com/paper/1809.02079/) (2018)
- [Reasoning Visual Dialogs with Structural and Partial Observations](https://mlsanity.com/paper/1904.05548/) (2019)
- [What You See is What You Get: Visual Pronoun Coreference Resolution in Dialogues](https://mlsanity.com/paper/1909.00421/) (2019)
- [Two can play this Game: Visual Dialog with Discriminative Question Generation and Answering](https://mlsanity.com/paper/1803.11186/) (2018)

## Abstract (arXiv)

Adversarial robustness evaluates the worst-case performance scenario of a machine learning model to ensure its safety and reliability. This study is the first to investigate the robustness of visually grounded dialog models towards textual attacks. These attacks represent a worst-case scenario where the input question contains a synonym which causes the previously correct model to return a wrong answer. Using this scenario, we first aim to understand how multimodal input components contribute to model robustness. Our results show that models which encode dialog history are more robust, and when launching an attack on history, model prediction becomes more uncertain. This is in contrast to prior work which finds that dialog history is negligible for model performance on this task. We also evaluate how to generate adversarial test examples which successfully fool the model but remain undetected by the user/software designer. We find that the textual, as well as the visual context are important to generate plausible worst-case scenarios.