# Why Did the Chicken Cross the Road? Rephrasing and Analyzing Ambiguous Questions in VQA

arXiv: https://arxiv.org/abs/2211.07516 · HTML: https://mlsanity.com/paper/2211.07516/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 48 references that are in the map.
- First posted on arXiv 14 Nov 2022.

## Builds on (most cited first, 17 of 17)

- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering](https://mlsanity.com/paper/1612.00837/) (2016)
- [CIDEr: Consensus-based Image Description Evaluation](https://mlsanity.com/paper/1411.5726/) (2014)
- [ViLT: Vision-and-Language Transformer Without Convolution or Region Supervision](https://mlsanity.com/paper/2102.03334/) (2021)
- [VizWiz Grand Challenge: Answering Visual Questions from Blind People](https://mlsanity.com/paper/1802.08218/) (2018)
- [AmbigQA: Answering Ambiguous Open-domain Questions](https://mlsanity.com/paper/2004.10645/) (2020)
- [Fast Lexically Constrained Decoding with Dynamic Beam Allocation for Neural Machine Translation](https://mlsanity.com/paper/1804.06609/) (2018)
- [SituatedQA: Incorporating Extra-Linguistic Contexts into QA](https://mlsanity.com/paper/2109.06157/) (2021)
- [Cycle-Consistency for Robust Visual Question Answering](https://mlsanity.com/paper/1902.05660/) (2019)
- [Generating Natural Questions About an Image](https://mlsanity.com/paper/1603.06059/) (2016)
- [Visual Question Generation as Dual Task of Visual Question Answering](https://mlsanity.com/paper/1709.07192/) (2017)
- [Information Maximizing Visual Question Generation](https://mlsanity.com/paper/1903.11207/) (2019)
- [Why Does a Visual Question Have Different Answers?](https://mlsanity.com/paper/1908.04342/) (2019)
- [iVQA: Inverse Visual Question Answering](https://mlsanity.com/paper/1710.03370/) (2017)
- [Guiding Visual Question Generation](https://mlsanity.com/paper/2110.08226/) (2021)
- [Rephrasing visual questions by specifying the entropy of the answer distribution](https://mlsanity.com/paper/2004.04963/) (2020)

## Built on by (most cited first, 13 of 13)

- [We're Afraid Language Models Aren't Modeling Ambiguity](https://mlsanity.com/paper/2304.14399/) (2023)
- [Zero and Few-shot Semantic Parsing with Ambiguous Inputs](https://mlsanity.com/paper/2306.00824/) (2023)
- [VQA Therapy: Exploring Answer Differences by Visually Grounding Answers](https://mlsanity.com/paper/2308.11662/) (2023)
- [Rephrase, Augment, Reason: Visual Grounding of Questions for Vision-Language Models](https://mlsanity.com/paper/2310.05861/) (2023)
- [Teaching Vision-Language Models to Ask: Resolving Ambiguity in Visual Questions](https://mlsanity.com/paper/2507.13773/) (2025)
- [RAcQUEt: Unveiling the Dangers of Overlooked Referential Ambiguity in Visual LLMs](https://mlsanity.com/paper/2412.13835/) (2024)
- [Acknowledging Focus Ambiguity in Visual Questions](https://mlsanity.com/paper/2501.02201/) (2025)
- [That was the last straw, we need more: Are Translation Systems Sensitive to Disambiguating Context?](https://mlsanity.com/paper/2310.14610/) (2023)
- [Trick or Neat: Adversarial Ambiguity and Language Model Evaluation](https://mlsanity.com/paper/2506.01205/) (2025)
- [Language Models Identify Ambiguities and Exploit Loopholes](https://mlsanity.com/paper/2508.19546/) (2025)
- [SimpsonsVQA: Enhancing Inquiry-Based Learning with a Tailored Dataset](https://mlsanity.com/paper/2410.22648/) (2024)
- [LaViSA: A Language and Vision Structural Ambiguity Benchmark](https://mlsanity.com/paper/2606.19552/) (2026)
- [CRAG-MM-Diagnostics: Enabling Stage-Wise Analysis of Knowledge-Intensive VQA](https://mlsanity.com/paper/2607.21155/) (2026)

## Abstract (arXiv)

Natural language is ambiguous. Resolving ambiguous questions is key to successfully answering them. Focusing on questions about images, we create a dataset of ambiguous examples. We annotate these, grouping answers by the underlying question they address and rephrasing the question for each group to reduce ambiguity. Our analysis reveals a linguistically-aligned ontology of reasons for ambiguity in visual questions. We then develop an English question-generation model which we demonstrate via automatic and human evaluation produces less ambiguous questions. We further show that the question generation objective we use allows the model to integrate answer group information without any direct supervision.