# Cross-Dataset Adaptation for Visual Question Answering

arXiv: https://arxiv.org/abs/1806.03726 · HTML: https://mlsanity.com/paper/1806.03726/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Jun 2018.

## Built on by (most cited first, 17 of 17)

- [Super-CLEVR: A Virtual Benchmark to Diagnose Domain Robustness in Visual Reasoning](https://mlsanity.com/paper/2212.00259/) (2022)
- [All You May Need for VQA are Image Captions](https://mlsanity.com/paper/2205.01883/) (2022)
- [Multimodal Co-learning: Challenges, Applications with Datasets, Recent Advances and Future Directions](https://mlsanity.com/paper/2107.13782/) (2021)
- [From Image to Language: A Critical Analysis of Visual Question Answering (VQA) Approaches, Challenges, and Opportunities](https://mlsanity.com/paper/2311.00308/) (2023)
- [Reassessing Evaluation Practices in Visual Question Answering: A Case Study on Out-of-Distribution Generalization](https://mlsanity.com/paper/2205.12191/) (2022)
- [Q: How to Specialize Large Vision-Language Models to Data-Scarce VQA Tasks? A: Self-Train on Unlabeled Images!](https://mlsanity.com/paper/2306.03932/) (2023)
- [Discovering the Unknown Knowns: Turning Implicit Knowledge in the Dataset into Explicit Training Examples for Visual Question Answering](https://mlsanity.com/paper/2109.06122/) (2021)
- [Domain-robust VQA with diverse datasets and methods but no target labels](https://mlsanity.com/paper/2103.15974/) (2021)
- [Transferring Domain-Agnostic Knowledge in Video Question Answering](https://mlsanity.com/paper/2110.13395/) (2021)
- [CL-CrossVQA: A Continual Learning Benchmark for Cross-Domain Visual Question Answering](https://mlsanity.com/paper/2211.10567/) (2022)
- [VideoDistill: Language-aware Vision Distillation for Video Question Answering](https://mlsanity.com/paper/2404.00973/) (2024)
- [DATE: Domain Adaptive Product Seeker for E-commerce](https://mlsanity.com/paper/2304.03669/) (2023)
- [From Recognition to Reasoning: Advancing Multimodal Harmful Meme Detection via Chain-of-Thought Alignment](https://mlsanity.com/paper/2506.18919/) (2025)
- [Question Generation for Evaluating Cross-Dataset Shifts in Multi-modal Grounding](https://mlsanity.com/paper/2201.09639/) (2022)
- [Modularized Zero-shot VQA with Pre-trained Models](https://mlsanity.com/paper/2305.17369/) (2023)
- [Disjoint Contrastive Regression Learning for Multi-Sourced Annotations](https://mlsanity.com/paper/2112.15411/) (2021)
- [What Are We Really Measuring? Rethinking Dataset Bias in Web-Scale Natural Image Collections via Unsupervised Semantic Clustering](https://mlsanity.com/paper/2604.13610/) (2026)

## Abstract (arXiv)

We investigate the problem of cross-dataset adaptation for visual question answering (Visual QA). Our goal is to train a Visual QA model on a source dataset but apply it to another target one. Analogous to domain adaptation for visual recognition, this setting is appealing when the target dataset does not have a sufficient amount of labeled data to learn an "in-domain" model. The key challenge is that the two datasets are constructed differently, resulting in the cross-dataset mismatch on images, questions, or answers. We overcome this difficulty by proposing a novel domain adaptation algorithm. Our method reduces the difference in statistical distributions by transforming the feature representation of the data in the target dataset. Moreover, it maximizes the likelihood of answering questions (in the target dataset) correctly using the Visual QA model trained on the source dataset. We empirically studied the effectiveness of the proposed approach on adapting among several popular Visual QA datasets. We show that the proposed method improves over baselines where there is no adaptation and several other adaptation methods. We both quantitatively and qualitatively analyze when the adaptation can be mostly effective.