# Semantic Equivalent Adversarial Data Augmentation for Visual Question Answering

arXiv: https://arxiv.org/abs/2007.09592 · HTML: https://mlsanity.com/paper/2007.09592/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Jul 2020.

## Built on by (most cited first, 26 of 26)

- [A Survey of Data Augmentation Approaches for NLP](https://mlsanity.com/paper/2105.03075/) (2021)
- [Teaching Structured Vision&Language Concepts to Vision&Language Models](https://mlsanity.com/paper/2211.11733/) (2022)
- [MixGen: A New Multi-Modal Data Augmentation](https://mlsanity.com/paper/2206.08358/) (2022)
- [Multi-task Paired Masking with Alignment Modeling for Medical Vision-Language Pre-training](https://mlsanity.com/paper/2305.07920/) (2023)
- [InstructTA: Instruction-Tuned Targeted Attack for Large Vision-Language Models](https://mlsanity.com/paper/2312.01886/) (2023)
- [Iterative Shrinking for Referring Expression Grounding Using Deep Reinforcement Learning](https://mlsanity.com/paper/2103.05187/) (2021)
- [Rethinking Data Augmentation for Robust Visual Question Answering](https://mlsanity.com/paper/2207.08739/) (2022)
- [Learning Multimodal Data Augmentation in Feature Space](https://mlsanity.com/paper/2212.14453/) (2022)
- [Discovering the Unknown Knowns: Turning Implicit Knowledge in the Dataset into Explicit Training Examples for Visual Question Answering](https://mlsanity.com/paper/2109.06122/) (2021)
- [TAG: Boosting Text-VQA via Text-aware Visual Question-answer Generation](https://mlsanity.com/paper/2208.01813/) (2022)
- [A survey on VQA_Datasets and Approaches](https://mlsanity.com/paper/2105.00421/) (2021)
- [SimVQA: Exploring Simulated Environments for Visual Question Answering](https://mlsanity.com/paper/2203.17219/) (2022)
- [Cross-Modal Generative Augmentation for Visual Question Answering](https://mlsanity.com/paper/2105.04780/) (2021)
- [3VL: Using Trees to Improve Vision-Language Models' Interpretability](https://mlsanity.com/paper/2312.17345/) (2023)
- [UPME: An Unsupervised Peer Review Framework for Multimodal Large Language Model Evaluation](https://mlsanity.com/paper/2503.14941/) (2025)
- [Multi-Modal Representation Learning with Text-Driven Soft Masks](https://mlsanity.com/paper/2304.00719/) (2023)
- [Bilateral Cross-Modality Graph Matching Attention for Feature Fusion in Visual Question Answering](https://mlsanity.com/paper/2112.07270/) (2021)
- [Enhancing Vietnamese VQA through Curriculum Learning on Raw and Augmented Text Representations](https://mlsanity.com/paper/2503.03285/) (2025)
- [Semantic Representation Attack against Aligned Large Language Models](https://mlsanity.com/paper/2509.19360/) (2025)
- [Show Why the Answer is Correct! Towards Explainable AI using Compositional Temporal Attention](https://mlsanity.com/paper/2105.07141/) (2021)
- [Improving Data Augmentation for Robust Visual Question Answering with Effective Curriculum Learning](https://mlsanity.com/paper/2401.15646/) (2024)
- [Harnessing Shared Relations via Multimodal Mixup Contrastive Learning for Multimodal Classification](https://mlsanity.com/paper/2409.17777/) (2024)
- [Contrastive Embedding Distribution Refinement and Entropy-Aware Attention for 3D Point Cloud Classification](https://mlsanity.com/paper/2201.11388/) (2022)
- [From text to multimodal: a survey of adversarial example generation in question answering systems](https://mlsanity.com/paper/2312.16156/) (2023)
- [GenView++: Unifying Adaptive Generative Augmentation and Quality-Driven Supervision for Contrastive Representation Learning](https://mlsanity.com/paper/2509.23770/) (2025)
- [LLM-Agnostic Semantic Representation Attack](https://mlsanity.com/paper/2605.08898/) (2026)

## Abstract (arXiv)

Visual Question Answering (VQA) has achieved great success thanks to the fast development of deep neural networks (DNN). On the other hand, the data augmentation, as one of the major tricks for DNN, has been widely used in many computer vision tasks. However, there are few works studying the data augmentation problem for VQA and none of the existing image based augmentation schemes (such as rotation and flipping) can be directly applied to VQA due to its semantic structure -- an langle image, question, answerrangle triplet needs to be maintained correctly. For example, a direction related Question-Answer (QA) pair may not be true if the associated image is rotated or flipped. In this paper, instead of directly manipulating images and questions, we use generated adversarial examples for both images and questions as the augmented data. The augmented examples do not change the visual properties presented in the image as well as the semantic meaning of the question, the correctness of the langle image, question, answerrangle is thus still maintained. We then use adversarial learning to train a classic VQA model (BUTD) with our augmented data. We find that we not only improve the overall performance on VQAv2, but also can withstand adversarial attack effectively, compared to the baseline model. The source code is available at https://github.com/zaynmi/seada-vqa.