# Generating Natural Questions About an Image

arXiv: https://arxiv.org/abs/1603.06059 · HTML: https://mlsanity.com/paper/1603.06059/

## Key facts

- Cited by 56 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Mar 2016.

## Built on by (most cited first, 30 of 56)

- [ChatGPT Asks, BLIP-2 Answers: Automatic Questioning Towards Enriched Visual Descriptions](https://mlsanity.com/paper/2303.06594/) (2023)
- [VIGC: Visual Instruction Generation and Correction](https://mlsanity.com/paper/2308.12714/) (2023)
- [All You May Need for VQA are Image Captions](https://mlsanity.com/paper/2205.01883/) (2022)
- [Learning to Answer Visual Questions from Web Videos](https://mlsanity.com/paper/2205.05019/) (2022)
- [Constructing Multi-Modal Dialogue Dataset by Replacing Text with Semantically Relevant Images](https://mlsanity.com/paper/2107.08685/) (2021)
- [From Image to Language: A Critical Analysis of Visual Question Answering (VQA) Approaches, Challenges, and Opportunities](https://mlsanity.com/paper/2311.00308/) (2023)
- [Quiz-Style Question Generation for News Stories](https://mlsanity.com/paper/2102.09094/) (2021)
- [LOVA3: Learning to Visual Question Answering, Asking and Assessment](https://mlsanity.com/paper/2405.14974/) (2024)
- [Question Generation for Reading Comprehension Assessment by Modeling How and What to Ask](https://mlsanity.com/paper/2204.02908/) (2022)
- [Q: How to Specialize Large Vision-Language Models to Data-Scarce VQA Tasks? A: Self-Train on Unlabeled Images!](https://mlsanity.com/paper/2306.03932/) (2023)
- [DREAM: Improving Situational QA by First Elaborating the Situation](https://mlsanity.com/paper/2112.08656/) (2021)
- [NAREOR: The Narrative Reordering Problem](https://mlsanity.com/paper/2104.06669/) (2021)
- [Why Did the Chicken Cross the Road? Rephrasing and Analyzing Ambiguous Questions in VQA](https://mlsanity.com/paper/2211.07516/) (2022)
- [A Framework for Learning to Request Rich and Contextually Useful Information from Humans](https://mlsanity.com/paper/2110.08258/) (2021)
- [Simple Baselines for Interactive Video Retrieval with Questions and Answers](https://mlsanity.com/paper/2308.10402/) (2023)
- [K-VQG: Knowledge-aware Visual Question Generation for Common-sense Acquisition](https://mlsanity.com/paper/2203.07890/) (2022)
- [Improving Visual Question Answering Models through Robustness Analysis and In-Context Learning with a Chain of Basic Questions](https://mlsanity.com/paper/2304.03147/) (2023)
- [Back-Training excels Self-Training at Unsupervised Domain Adaptation of Question Generation and Passage Retrieval](https://mlsanity.com/paper/2104.08801/) (2021)
- [Cross-Modal Generative Augmentation for Visual Question Answering](https://mlsanity.com/paper/2105.04780/) (2021)
- [QAVA: Query-Agnostic Visual Attack to Large Vision-Language Models](https://mlsanity.com/paper/2504.11038/) (2025)
- [Iconary: A Pictionary-Based Game for Testing Multimodal Communication with Drawings and Text](https://mlsanity.com/paper/2112.00800/) (2021)
- [Look, Read and Ask: Learning to Ask Questions by Reading Text in Images](https://mlsanity.com/paper/2211.12950/) (2022)
- [Proactive and Reactive Engagement of Artificial Intelligence Methods for Education: A Review](https://mlsanity.com/paper/2301.10231/) (2023)
- [A Survey on Neural Question Generation: Methods, Applications, and Prospects](https://mlsanity.com/paper/2402.18267/) (2024)
- [Words of Wisdom: Representational Harms in Learning From AI Communication](https://mlsanity.com/paper/2111.08581/) (2021)
- [Location-Aware Visual Question Generation with Lightweight Models](https://mlsanity.com/paper/2310.15129/) (2023)
- [ConVQG: Contrastive Visual Question Generation with Multimodal Guidance](https://mlsanity.com/paper/2402.12846/) (2024)
- [ELBA: Learning by Asking for Embodied Visual Navigation and Task Completion](https://mlsanity.com/paper/2302.04865/) (2023)
- [Enhancing Vision-Language Pre-Training with Jointly Learned Questioner and Dense Captioner](https://mlsanity.com/paper/2305.11769/) (2023)
- [Enhancing Question Generation with Commonsense Knowledge](https://mlsanity.com/paper/2106.10454/) (2021)

## Abstract (arXiv)

There has been an explosion of work in the vision & language community during the past few years from image captioning to video transcription, and answering questions about images. These tasks have focused on literal descriptions of the image. To move beyond the literal, we choose to explore how questions about an image are often directed at commonsense inference and the abstract events evoked by objects in the image. In this paper, we introduce the novel task of Visual Question Generation (VQG), where the system is tasked with asking a natural and engaging question when shown an image. We provide three datasets which cover a variety of images from object-centric to event-centric, with considerably more abstract training data than provided to state-of-the-art captioning systems thus far. We train and test several generative and retrieval models to tackle the task of VQG. Evaluation results show that while such models ask reasonable questions for a variety of images, there is still a wide gap with human performance which motivates further work on connecting images with commonsense knowledge and pragmatics. Our proposed task offers a new challenge to the community which we hope furthers interest in exploring deeper connections between vision & language.