# Zero-Shot Visual Question Answering

arXiv: https://arxiv.org/abs/1611.05546 · HTML: https://mlsanity.com/paper/1611.05546/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Nov 2016.

## Built on by (most cited first, 20 of 20)

- [All You May Need for VQA are Image Captions](https://mlsanity.com/paper/2205.01883/) (2022)
- [Knowledge-aware Zero-Shot Learning: Survey and Perspective](https://mlsanity.com/paper/2103.00070/) (2021)
- [Exemplar Guided Deep Neural Network for Spatial Transcriptomics Analysis of Gene Expression Prediction](https://mlsanity.com/paper/2210.16721/) (2022)
- [Zero-shot Visual Question Answering using Knowledge Graph](https://mlsanity.com/paper/2107.05348/) (2021)
- [Pseudo-Q: Generating Pseudo Language Queries for Visual Grounding](https://mlsanity.com/paper/2203.08481/) (2022)
- [CLIP-TD: CLIP Targeted Distillation for Vision-Language Tasks](https://mlsanity.com/paper/2201.05729/) (2022)
- [Learning to Answer Visual Questions from Web Videos](https://mlsanity.com/paper/2205.05019/) (2022)
- [From Image to Language: A Critical Analysis of Visual Question Answering (VQA) Approaches, Challenges, and Opportunities](https://mlsanity.com/paper/2311.00308/) (2023)
- [Reasoning over Vision and Language: Exploring the Benefits of Supplemental Knowledge](https://mlsanity.com/paper/2101.06013/) (2021)
- [Separating Skills and Concepts for Novel Visual Question Answering](https://mlsanity.com/paper/2107.09106/) (2021)
- [Zero-shot and Few-shot Learning with Knowledge Graphs: A Comprehensive Survey](https://mlsanity.com/paper/2112.10006/) (2021)
- [Multimodal Adaptive Distillation for Leveraging Unimodal Encoders for Vision-Language Tasks](https://mlsanity.com/paper/2204.10496/) (2022)
- [Visual Robustness Benchmark for Visual Question Answering (VQA)](https://mlsanity.com/paper/2407.03386/) (2024)
- [Cross3DVG: Cross-Dataset 3D Visual Grounding on Different RGB-D Scans](https://mlsanity.com/paper/2305.13876/) (2023)
- [Divide and Conquer: Answering Questions with Object Factorization and Compositional Reasoning](https://mlsanity.com/paper/2303.10482/) (2023)
- [Mixture of Rationale: Multi-Modal Reasoning Mixture for Visual Question Answering](https://mlsanity.com/paper/2406.01402/) (2024)
- [Improved Few-Shot Image Classification Through Multiple-Choice Questions](https://mlsanity.com/paper/2407.16145/) (2024)
- [Few-Shot Visual Question Generation: A Novel Task and Benchmark Datasets](https://mlsanity.com/paper/2210.07076/) (2022)
- [On the Cognition of Visual Question Answering Models and Human Intelligence: A Comparative Study](https://mlsanity.com/paper/2310.02528/) (2023)
- [QTG-VQA: Question-Type-Guided Architectural for VideoQA Systems](https://mlsanity.com/paper/2409.09348/) (2024)

## Abstract (arXiv)

Part of the appeal of Visual Question Answering (VQA) is its promise to answer new questions about previously unseen images. Most current methods demand training questions that illustrate every possible concept, and will therefore never achieve this capability, since the volume of required training data would be prohibitive. Answering general questions about images requires methods capable of Zero-Shot VQA, that is, methods able to answer questions beyond the scope of the training questions. We propose a new evaluation protocol for VQA methods which measures their ability to perform Zero-Shot VQA, and in doing so highlights significant practical deficiencies of current approaches, some of which are masked by the biases in current datasets. We propose and evaluate several strategies for achieving Zero-Shot VQA, including methods based on pretrained word embeddings, object classifiers with semantic embeddings, and test-time retrieval of example images. Our extensive experiments are intended to serve as baselines for Zero-Shot VQA, and they also achieve state-of-the-art performance in the standard VQA evaluation setting.