# VQA-GEN: A Visual Question Answering Benchmark for Domain Generalization

arXiv: https://arxiv.org/abs/2311.00807 · HTML: https://mlsanity.com/paper/2311.00807/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal reasoning, one of 17 directions in LLM reasoning.
- Builds on 23 of its 27 references that are in the map.
- First posted on arXiv 1 Nov 2023.

## Placed in

- LLM reasoning / [Multimodal reasoning](https://mlsanity.com/lr/multimodal-reasoning/)

## Builds on (most cited first, 23 of 23)

- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations](https://mlsanity.com/paper/1602.07332/) (2016)
- [VQA: Visual Question Answering](https://mlsanity.com/paper/1505.00468/) (2015)
- [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization](https://mlsanity.com/paper/1703.06868/) (2017)
- [CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning](https://mlsanity.com/paper/1612.06890/) (2016)
- [VizWiz Grand Challenge: Answering Visual Questions from Blind People](https://mlsanity.com/paper/1802.08218/) (2018)
- [Universal Style Transfer via Feature Transforms](https://mlsanity.com/paper/1705.08086/) (2017)
- [Don't Just Assume; Look and Answer: Overcoming Priors for Visual Question Answering](https://mlsanity.com/paper/1712.00377/) (2017)
- [Visual7W: Grounded Question Answering in Images](https://mlsanity.com/paper/1511.03416/) (2015)
- [Style Transfer from Non-Parallel Text by Cross-Alignment](https://mlsanity.com/paper/1705.09655/) (2017)
- [Marian: Fast Neural Machine Translation in C++](https://mlsanity.com/paper/1804.00344/) (2018)
- [Exploring Models and Data for Image Question Answering](https://mlsanity.com/paper/1505.02074/) (2015)
- [Domain Enhanced Arbitrary Image Style Transfer via Contrastive Learning](https://mlsanity.com/paper/2205.09542/) (2022)
- [VQA-LOL: Visual Question Answering under the Lens of Logic](https://mlsanity.com/paper/2002.08325/) (2020)
- [DoCoGen: Domain Counterfactual Generation for Low Resource Domain Adaptation](https://mlsanity.com/paper/2202.12350/) (2022)
- [Multi-Task Neural Models for Translating Between Styles Within and Across Languages](https://mlsanity.com/paper/1806.04357/) (2018)
- [Thank you BART! Rewarding Pre-Trained Models Improves Formality Style Transfer](https://mlsanity.com/paper/2105.06947/) (2021)
- [A Hierarchical Reinforced Sequence Operation Method for Unsupervised Text Style Transfer](https://mlsanity.com/paper/1906.01833/) (2019)
- [Semi-supervised Formality Style Transfer using Language Model Discriminator and Mutual Information Maximization](https://mlsanity.com/paper/2010.05090/) (2020)
- [Discovering the Unknown Knowns: Turning Implicit Knowledge in the Dataset into Explicit Training Examples for Visual Question Answering](https://mlsanity.com/paper/2109.06122/) (2021)
- [Reinforced Rewards Framework for Text Style Transfer](https://mlsanity.com/paper/2005.05256/) (2020)
- [Multidimensional Evaluation for Text Style Transfer Using ChatGPT](https://mlsanity.com/paper/2304.13462/) (2023)

## Built on by (most cited first, 5 of 5)

- [EgoNight: Towards Egocentric Vision Understanding at Night with a Challenging Benchmark](https://mlsanity.com/paper/2510.06218/) (2025)
- [Improving Generalization in Visual Reasoning via Self-Ensemble](https://mlsanity.com/paper/2410.20883/) (2024)
- [FRAMES-VQA: Benchmarking Fine-Tuning Robustness across Multi-Modal Shifts in Visual Question Answering](https://mlsanity.com/paper/2505.21755/) (2025)
- [VolDoGer: LLM-assisted Datasets for Domain Generalization in Vision-Language Tasks](https://mlsanity.com/paper/2407.19795/) (2024)
- [Text-conditioned State Space Model For Domain-generalized Change Detection Visual Question Answering](https://mlsanity.com/paper/2508.08974/) (2025)

## Abstract (arXiv)

Visual question answering (VQA) models are designed to demonstrate visual-textual reasoning capabilities. However, their real-world applicability is hindered by a lack of comprehensive benchmark datasets. Existing domain generalization datasets for VQA exhibit a unilateral focus on textual shifts while VQA being a multi-modal task contains shifts across both visual and textual domains. We propose VQA-GEN, the first ever multi-modal benchmark dataset for distribution shift generated through a shift induced pipeline. Experiments demonstrate VQA-GEN dataset exposes the vulnerability of existing methods to joint multi-modal distribution shifts. validating that comprehensive multi-modal shifts are critical for robust VQA generalization. Models trained on VQA-GEN exhibit improved cross-domain and in-domain performance, confirming the value of VQA-GEN. Further, we analyze the importance of each shift technique of our pipeline contributing to the generalization of the model.