# Rethinking Data Augmentation for Robust Visual Question Answering

arXiv: https://arxiv.org/abs/2207.08739 · HTML: https://mlsanity.com/paper/2207.08739/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal distillation, one of 12 directions in Knowledge distillation.
- Builds on 44 of its 63 references that are in the map.
- First posted on arXiv 18 Jul 2022.

## Placed in

- Knowledge distillation / [Multimodal distillation](https://mlsanity.com/distill/multimodal-distillation/)

## Builds on (most cited first, 30 of 44)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations](https://mlsanity.com/paper/1602.07332/) (2016)
- [Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering](https://mlsanity.com/paper/1612.00837/) (2016)
- [VQA: Visual Question Answering](https://mlsanity.com/paper/1505.00468/) (2015)
- [CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning](https://mlsanity.com/paper/1612.06890/) (2016)
- [Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering](https://mlsanity.com/paper/1707.07998/) (2017)
- [Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Books](https://mlsanity.com/paper/1506.06724/) (2015)
- [Don't Just Assume; Look and Answer: Overcoming Priors for Visual Question Answering](https://mlsanity.com/paper/1712.00377/) (2017)
- [Knowledge Distillation and Student-Teacher Learning for Visual Intelligence: A Review and New Outlooks](https://mlsanity.com/paper/2004.05937/) (2020)
- [Don't Take the Easy Way Out: Ensemble Based Methods for Avoiding Known Dataset Biases](https://mlsanity.com/paper/1909.03683/) (2019)
- [Counterfactual VQA: A Cause-Effect Look at Language Bias](https://mlsanity.com/paper/2006.04315/) (2020)
- [SCA-CNN: Spatial and Channel-wise Attention in Convolutional Networks for Image Captioning](https://mlsanity.com/paper/1611.05594/) (2016)
- [Distilling Object Detectors with Fine-grained Feature Imitation](https://mlsanity.com/paper/1906.03609/) (2019)
- [RUBi: Reducing Unimodal Biases in Visual Question Answering](https://mlsanity.com/paper/1906.10169/) (2019)
- [Counterfactual Samples Synthesizing for Robust Visual Question Answering](https://mlsanity.com/paper/2003.06576/) (2020)
- [End-to-End Bias Mitigation by Modelling Biases in Corpora](https://mlsanity.com/paper/1909.06321/) (2019)
- [Data Distillation: Towards Omni-Supervised Learning](https://mlsanity.com/paper/1712.04440/) (2017)
- [Analyzing the Behavior of Visual Question Answering Models](https://mlsanity.com/paper/1606.07356/) (2016)
- [Yin and Yang: Balancing and Answering Binary Visual Questions](https://mlsanity.com/paper/1511.05099/) (2015)
- [Spatio-Temporal Graph for Video Captioning with Knowledge Distillation](https://mlsanity.com/paper/2003.13942/) (2020)
- [Towards Causal VQA: Revealing and Reducing Spurious Correlations by Invariant and Covariant Semantic Editing](https://mlsanity.com/paper/1912.07538/) (2019)
- [Object Relational Graph with Teacher-Recommended Learning for Video Captioning](https://mlsanity.com/paper/2002.11566/) (2020)
- [Overcoming Language Priors in Visual Question Answering with Adversarial Regularization](https://mlsanity.com/paper/1810.03649/) (2018)
- [Boundary Proposal Network for Two-Stage Natural Language Video Localization](https://mlsanity.com/paper/2103.08109/) (2021)
- [On the Value of Out-of-Distribution Testing: An Example of Goodhart's Law](https://mlsanity.com/paper/2005.09241/) (2020)
- [MUREL: Multimodal Relational Reasoning for Visual Question Answering](https://mlsanity.com/paper/1902.09487/) (2019)
- [MUTANT: A Training Paradigm for Out-of-Distribution Generalization in Visual Question Answering](https://mlsanity.com/paper/2009.08566/) (2020)

## Built on by (most cited first, 18 of 18)

- [Context-Guided Spatio-Temporal Video Grounding](https://mlsanity.com/paper/2401.01578/) (2024)
- [SUR-adapter: Enhancing Text-to-Image Pre-trained Diffusion Models with Large Language Models](https://mlsanity.com/paper/2305.05189/) (2023)
- [Robust Visual Question Answering: Datasets, Methods, and Future Challenges](https://mlsanity.com/paper/2307.11471/) (2023)
- [Integrating Object-aware and Interaction-aware Knowledge for Weakly Supervised Scene Graph Generation](https://mlsanity.com/paper/2208.01834/) (2022)
- [NICEST: Noisy Label Correction and Training for Robust Scene Graph Generation](https://mlsanity.com/paper/2207.13316/) (2022)
- [Q: How to Specialize Large Vision-Language Models to Data-Scarce VQA Tasks? A: Self-Train on Unlabeled Images!](https://mlsanity.com/paper/2306.03932/) (2023)
- [Look, Listen, and Answer: Overcoming Biases for Audio-Visual Question Answering](https://mlsanity.com/paper/2404.12020/) (2024)
- [Label Semantic Knowledge Distillation for Unbiased Scene Graph Generation](https://mlsanity.com/paper/2208.03763/) (2022)
- [Generative Bias for Robust Visual Question Answering](https://mlsanity.com/paper/2208.00690/) (2022)
- [Correspondence Matters for Video Referring Expression Comprehension](https://mlsanity.com/paper/2207.10400/) (2022)
- [Debiasing Multimodal Models via Causal Information Minimization](https://mlsanity.com/paper/2311.16941/) (2023)
- [Eliminating the Language Bias for Visual Question Answering with fine-grained Causal Intervention](https://mlsanity.com/paper/2410.10184/) (2024)
- [Automatic Diagnosis of Myocarditis Disease in Cardiac MRI Modality using Deep Transformers and Explainable Artificial Intelligence](https://mlsanity.com/paper/2210.14611/) (2022)
- [Adaptive loose optimization for robust question answering](https://mlsanity.com/paper/2305.03971/) (2023)
- ['No' Matters: Out-of-Distribution Detection in Multimodality Long Dialogue](https://mlsanity.com/paper/2410.23883/) (2024)
- [Overcoming Language Priors for Visual Question Answering Based on Knowledge Distillation](https://mlsanity.com/paper/2501.05690/) (2025)
- [Integrating Object Interaction Self-Attention and GAN-Based Debiasing for Visual Question Answering](https://mlsanity.com/paper/2509.20884/) (2025)
- [Counterfactual Reasoning for Robust Visual Question Answering](https://mlsanity.com/paper/2609.16567/) (2026)

## Abstract (arXiv)

Data Augmentation (DA) -- generating extra training samples beyond original training set -- has been widely-used in today's unbiased VQA models to mitigate the language biases. Current mainstream DA strategies are synthetic-based methods, which synthesize new samples by either editing some visual regions/words, or re-generating them from scratch. However, these synthetic samples are always unnatural and error-prone. To avoid this issue, a recent DA work composes new augmented samples by randomly pairing pristine images and other human-written questions. Unfortunately, to guarantee augmented samples have reasonable ground-truth answers, they manually design a set of heuristic rules for several question types, which extremely limits its generalization abilities. To this end, we propose a new Knowledge Distillation based Data Augmentation for VQA, dubbed KDDAug. Specifically, we first relax the requirements of reasonable image-question pairs, which can be easily applied to any question types. Then, we design a knowledge distillation (KD) based answer assignment to generate pseudo answers for all composed image-question pairs, which are robust to both in-domain and out-of-distribution settings. Since KDDAug is a model-agnostic DA strategy, it can be seamlessly incorporated into any VQA architectures. Extensive ablation studies on multiple backbones and benchmarks have demonstrated the effectiveness and generalization abilities of KDDAug.