# Generating Question Relevant Captions to Aid Visual Question Answering

arXiv: https://arxiv.org/abs/1906.00513 · HTML: https://mlsanity.com/paper/1906.00513/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Jun 2019.

## Built on by (most cited first, 10 of 10)

- [From Images to Textual Prompts: Zero-shot VQA with Frozen Large Language Models](https://mlsanity.com/paper/2212.10846/) (2022)
- [All You May Need for VQA are Image Captions](https://mlsanity.com/paper/2205.01883/) (2022)
- [Integrating Visuospatial, Linguistic and Commonsense Structure into Story Visualization](https://mlsanity.com/paper/2110.10834/) (2021)
- [LPF: A Language-Prior Feedback Objective Function for De-biased Visual Question Answering](https://mlsanity.com/paper/2105.14300/) (2021)
- [Discovering the Unknown Knowns: Turning Implicit Knowledge in the Dataset into Explicit Training Examples for Visual Question Answering](https://mlsanity.com/paper/2109.06122/) (2021)
- [Entity-Focused Dense Passage Retrieval for Outside-Knowledge Visual Question Answering](https://mlsanity.com/paper/2210.10176/) (2022)
- [Latent Variable Models for Visual Question Answering](https://mlsanity.com/paper/2101.06399/) (2021)
- [Visual Question Answering based on Local-Scene-Aware Referring Expression Generation](https://mlsanity.com/paper/2101.08978/) (2021)
- [Towards Reasoning-Aware Explainable VQA](https://mlsanity.com/paper/2211.05190/) (2022)
- [Beam-Guided Knowledge Replay for Knowledge-Rich Image Captioning using Vision-Language Model](https://mlsanity.com/paper/2505.23358/) (2025)

## Abstract (arXiv)

Visual question answering (VQA) and image captioning require a shared body of general knowledge connecting language and vision. We present a novel approach to improve VQA performance that exploits this connection by jointly generating captions that are targeted to help answer a specific visual question. The model is trained using an existing caption dataset by automatically determining question-relevant captions using an online gradient-based method. Experimental results on the VQA v2 challenge demonstrates that our approach obtains state-of-the-art VQA performance (e.g. 68.4% on the Test-standard set using a single model) by simultaneously generating question-relevant captions.