mlsanityData to 14 Oct 2026

Teaching Vision-Language Models to Ask: Resolving Ambiguity in Visual Questions

arXiv 2507.13773 ↗First posted 18 Jul 2025cs.CV cs.CLcited by 11

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning

Abstract (arXiv)

In visual question answering (VQA) context, users often pose ambiguous questions to visual language models (VLMs) due to varying expression habits. Existing research addresses such ambiguities primarily by rephrasing questions. These approaches neglect the inherently interactive nature of user interactions with VLMs, where ambiguities can be clarified through user feedback. However, research on interactive clarification faces two major challenges: (1) Benchmarks are absent to assess VLMs' capacity for resolving ambiguities through interaction; (2) VLMs are trained to prefer answering rather than asking, preventing them from seeking clarification. To overcome these challenges, we introduce ClearVQA benchmark, which targets three common categories of ambiguity in VQA context, and encompasses various VQA scenarios.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 36 of its 60 references that are in the map. First posted on arXiv 18 Jul 2025.

Builds on

36 of 60 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024Vision-Language Models Can Self-Improve Reasoning via Reflection30 Oct
  2. 2024MACAROON: Training Vision-Language Models To Be Your Engaged Partners20 Jun
  3. 2024Selective "Selective Prediction": Reducing Unnecessary Abstention in Vision-Language Reasoning23 Feb
  4. 2023Lyrics: Boosting Fine-grained Language-Vision Alignment and Comprehension via Semantic-aware Visual Objects8 Dec

Other references · 32

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  4. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  5. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
  6. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,876
  7. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,516
  8. 2015VQA: Visual Question Answeringcited by 2,506
  9. 2017Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answeringcited by 1,312
  10. 2019OK-VQA: A Visual Question Answering Benchmark Requiring External Knowledgecited by 1,121
  11. 2022A-OKVQA: A Benchmark for Visual Question Answering using World Knowledgecited by 710
  12. 2023The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)cited by 681
20 more
  1. 2020AmbigQA: Answering Ambiguous Open-domain Questionscited by 390
  2. 2016Visual Dialogcited by 377
  3. 2019Scene Text Visual Question Answeringcited by 331
  4. 2024AlphaMath Almost Zero: Process Supervision without Processcited by 162
  5. 2023Prompting and Evaluating Large Language Models for Proactive Dialogues: Clarification, Target-guided, and Non-collaborationcited by 116
  6. 2025Towards Scientific Intelligence: A Survey of LLM-based Scientific Agentscited by 98
  7. 2022Reliable Visual Question Answering: Abstain Rather Than Answer Incorrectlycited by 89
  8. 2015Explicit Knowledge-based Reasoning for Visual Question Answeringcited by 85
  9. 2024Towards Human-centered Proactive Conversational Agentscited by 49
  10. 2019Why Does a Visual Question Have Different Answers?cited by 34
  11. 2022Tackling Ambiguity with Images: Improved Multimodal Machine Translation and Contrastive Evaluationcited by 26
  12. 2021NoiseQA: Challenge Set Evaluation for User-Centric Question Answeringcited by 25
  13. 2023Improving Selective Visual Question Answering by Learning from Your Peerscited by 25
  14. 2025KTAE: A Model-Free Algorithm to Key-Tokens Advantage Estimation in Mathematical Reasoningcited by 19
  15. 2024Visual-O1: Understanding Ambiguous Instructions via Multi-modal Multi-turn Chain-of-thoughts Reasoningcited by 17
  16. 2025CROP: Contextual Region-Oriented Visual Token Pruningcited by 14
  17. 2022Why Did the Chicken Cross the Road? Rephrasing and Analyzing Ambiguous Questions in VQAcited by 13
  18. 2020F1 is Not Enough! Models and Evaluation Towards User-Centered Explainable Question Answeringcited by 11
  19. 2025From Generic Empathy to Personalized Emotional Support: A Self-Evolution Framework for User Preference Alignmentcited by 9
  20. 2024Navigating Brain Language Representations: A Comparative Analysis of Neural Language Models and Psychologically Plausible Modelscited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 10.

11 papers.

  1. 2026Clarify or Answer: Reinforcement Learning for Agentic VQA with Context Under-specificationcited by 3
  2. 2026What Users Leave Unsaid: Under-Specified Queries Limit Vision-Language Modelscited by 2
  3. 2025Discovering Semantic Subdimensions through Disentangled Conceptual Representationscited by 1
  4. 2026Towards Mitigating Modality Bias in Vision-Language Models for Temporal Action Localizationcited by 1
  5. 2026AQuA: Toward Strategic Response Generation for Ambiguous Visual Questionscited by 1
  6. 2026VIABLE: A Visually Impaired Assistance Benchmark for VLM-as-a-Judge Evaluationcited by 1
  7. 2026Seeing is not Enough: Vision-Language Models Perceive Evidence but Fail to Actcited by 1
  8. 2026PGT: Procedurally Generated Tasks for improving visual grounding in MLLMscited by 0
  9. 2026Uncertainty-Aware Decision Making in Multimodal Large Language Modelscited by 0
  10. 2026ManGo: Manga Active Narrative Grounding Optimizationcited by 0
  11. 2026WeaveAgent: A Two-Stage Tool-Routing Agent for Ultra-High-Resolution Remote Sensing Imagerycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Logical Implications for Visual Question Answering Consistency
  2. 2026Evaluating VQA in Vision Language Models using Cooperative Principles
  3. 2024Guiding Vision-Language Model Selection for Visual Question-Answering Across Tasks, Domains, and Knowledge Types
  4. 2025Evaluating Variance in Visual Question Answering Benchmarks
  5. 2025Questioning the Stability of Visual Question Answering
  6. 2024A Comprehensive Survey on Visual Question Answering Datasets and Algorithms
  7. 2025Exploring Advanced Techniques for Visual Question Answering: A Comprehensive Comparison
  8. 2024Natural Language Understanding and Inference with MLLM in Visual Question Answering: A Survey