# Open-Ended Visual Question Answering by Multi-Modal Domain Adaptation

arXiv: https://arxiv.org/abs/1911.04058 · HTML: https://mlsanity.com/paper/1911.04058/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Nov 2019.

## Built on by (most cited first, 12 of 12)

- [Super-CLEVR: A Virtual Benchmark to Diagnose Domain Robustness in Visual Reasoning](https://mlsanity.com/paper/2212.00259/) (2022)
- [Q: How to Specialize Large Vision-Language Models to Data-Scarce VQA Tasks? A: Self-Train on Unlabeled Images!](https://mlsanity.com/paper/2306.03932/) (2023)
- [Domain-robust VQA with diverse datasets and methods but no target labels](https://mlsanity.com/paper/2103.15974/) (2021)
- [Semantically Distributed Robust Optimization for Vision-and-Language Inference](https://mlsanity.com/paper/2110.07165/) (2021)
- [Transferring Domain-Agnostic Knowledge in Video Question Answering](https://mlsanity.com/paper/2110.13395/) (2021)
- [CL-CrossVQA: A Continual Learning Benchmark for Cross-Domain Visual Question Answering](https://mlsanity.com/paper/2211.10567/) (2022)
- [Towards Answering Open-ended Ethical Quandary Questions](https://mlsanity.com/paper/2205.05989/) (2022)
- [Modularized Zero-shot VQA with Pre-trained Models](https://mlsanity.com/paper/2305.17369/) (2023)
- [Hybrid-Learning Video Moment Retrieval across Multi-Domain Labels](https://mlsanity.com/paper/2406.01791/) (2024)
- [Text-conditioned State Space Model For Domain-generalized Change Detection Visual Question Answering](https://mlsanity.com/paper/2508.08974/) (2025)
- [APAM: Adaptive Pre-training and Adaptive Meta Learning in Language Model for Noisy Labels and Long-tailed Learning](https://mlsanity.com/paper/2302.03488/) (2023)
- [ECA: Efficient Continual Alignment for Open-Ended Image-to-Text Generation](https://mlsanity.com/paper/2606.12633/) (2026)

## Abstract (arXiv)

We study the problem of visual question answering (VQA) in images by exploiting supervised domain adaptation, where there is a large amount of labeled data in the source domain but only limited labeled data in the target domain with the goal to train a good target model. A straightforward solution is to fine-tune a pre-trained source model by using those limited labeled target data, but it usually cannot work well due to the considerable difference between the data distributions of the source and target domains. Moreover, the availability of multiple modalities (i.e., images, questions and answers) in VQA poses further challenges to model the transferability between those different modalities. In this paper, we tackle the above issues by proposing a novel supervised multi-modal domain adaptation method for VQA to learn joint feature embeddings across different domains and modalities. Specifically, we align the data distributions of the source and target domains by considering all modalities together as well as separately for each individual modality. Based on the extensive experiments on the benchmark VQA 2.0 and VizWiz datasets for the realistic open-ended VQA task, we demonstrate that our proposed method outperforms the existing state-of-the-art approaches in this challenging domain adaptation setting for VQA.