Grounding Answers for Visual Questions Asked by Visually Impaired People
Not on the map. Why?
Abstract (arXiv)
Visual question answering is the task of answering questions about images. We introduce the VizWiz-VQA-Grounding dataset, the first dataset that visually grounds answers to visual questions asked by people with visual impairments. We analyze our dataset and compare it with five VQA-Grounding datasets to demonstrate what makes it similar and different. We then evaluate the SOTA VQA and VQA-Grounding models and demonstrate that current SOTA algorithms often fail to identify the correct visual evidence where the answer is located. These models regularly struggle when the visual evidence occupies a small fraction of the image, for images that are higher quality, as well as for visual questions that require skills in text recognition. The dataset, evaluation server, and leaderboard all can be found at the following link: https://vizwiz.org/tasks-and-datasets/answer-grounding-for-vqa/.
Key facts. Cited by 47 papers in the map (data through 7 Oct 2026). Builds on 26 of its 42 references that are in the map. First posted on arXiv 4 Feb 2022.
Builds on
26 of 42 references.
References · 26
- 2014Microsoft COCO: Common Objects in Contextcited by 17,498
- 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,646
- 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,498
- 2015VQA: Visual Question Answeringcited by 2,489
- 2019Towards VQA Models That Can Readcited by 1,696
- 2019LXMERT: Learning Cross-Modality Encoder Representations from Transformerscited by 1,548
- 2020Oscar: Object-Semantics Aligned Pre-training for Vision-Language Taskscited by 1,213
- 2018VizWiz Grand Challenge: Answering Visual Questions from Blind Peoplecited by 852
- 2015Visual7W: Grounded Question Answering in Imagescited by 315
- 2020Counterfactual VQA: A Cause-Effect Look at Language Biascited by 266
- 2018Multimodal Explanations: Justifying Decisions and Pointing to the Evidencecited by 179
- 2016Visual Question Answering: A Survey of Methods and Datasetscited by 123
14 more
- 2016Human Attention in Visual Question Answering: Do Humans and Deep Networks Look at the Same Regions?cited by 111
- 2020Captioning Images Taken by People Who Are Blindcited by 108
- 2018Overcoming Language Priors in Visual Question Answering with Adversarial Regularizationcited by 104
- 2017An Analysis of Visual Question Answering Algorithmscited by 90
- 2017VQS: Linking Segmentations to Questions and Answers for Supervised Attention in VQA and Question-Focused Semantic Segmentationcited by 50
- 2017Exploring Human-like Attention Supervision in Visual Question Answeringcited by 36
- 2020Reducing Language Biases in Visual Question Answering with Visually-Grounded Question Encodercited by 34
- 2018Interpretable Visual Question Answering by Visual Grounding from Attention Supervision Miningcited by 28
- 2020Assessing Image Quality Issues for Real-World Problemscited by 28
- 2020AiR: Attention with Reasoning Capabilitycited by 28
- 2021Found a Reason for me? Weakly-supervised Grounded Visual Question Answering using Capsulescited by 22
- 2019Visual Question Answering using Deep Learning: A Survey and Performance Analysiscited by 19
- 2020Vision Skills Needed to Answer Visual Questionscited by 17
- 2021A First Look: Towards Explainable TextVQA Models via Visual and Textual Explanationscited by 14
Built on by
Citing papers per year: 2022: 5; 2023: 16; 2024: 12; 2025: 7; 2026: 7.
47 papers.
Sort:
By area:
- 2022Unified-IO: A Unified Model for Vision, Language, and Multi-Modal Taskscited by 420
- 2022Pix2Struct: Screenshot Parsing as Pretraining for Visual Language Understandingcited by 360
- 2022EPIC-KITCHENS VISOR Benchmark: VIdeo Segmentations and Object Relationscited by 139
- 2023Can I Trust Your Answer? Visually Grounded Video Question Answeringcited by 133
- 2023An Outlook into the Future of Egocentric Visioncited by 84
- 2024A Survey of Attacks on Large Vision-Language Models: Resources, Advances, and Future Trendscited by 82
- 2024GROUNDHOG: Grounding Large Language Models to Holistic Segmentationcited by 70
- 2024Toward Visual Grounding: A Surveycited by 52
- 2024MC-Bench: A Benchmark for Multi-Context Visual Grounding in the Era of MLLMscited by 38
- 2023From Image to Language: A Critical Analysis of Visual Question Answering (VQA) Approaches, Challenges, and Opportunitiescited by 24
- 2024OmniCount: Multi-label Object Counting with Semantic-Geometric Priorscited by 16
- 2023VQA Therapy: Exploring Answer Differences by Visually Grounding Answerscited by 14
35 more
- 2025Survey of Adversarial Robustness in Multimodal Large Language Modelscited by 14
- 2022VizWiz-FewShot: Locating Objects in Images Taken by People With Visual Impairmentscited by 11
- 2023Toloka Visual Question Answering Benchmarkcited by 10
- 2024Right this way: Can VLMs Guide Us to See More to Answer Questions?cited by 10
- 2025RefChartQA: Grounding Visual Answer on Chart Images through Instruction Tuningcited by 10
- 2023Dealing with Semantic Underspecification in Multimodal NLPcited by 9
- 2025Towards Understanding Visual Grounding in Visual Language Modelscited by 9
- 2023Quality-agnostic Image Captioning to Safely Assist People with Vision Impairmentcited by 8
- 2023Sentence Attention Blocks for Answer Groundingcited by 8
- 2023Salient Object Detection for Images Taken by People With Vision Impairmentscited by 7
- 2025Acknowledging Focus Ambiguity in Visual Questionscited by 7
- 2023Logical Implications for Visual Question Answering Consistencycited by 6
- 2024On the Role of Visual Grounding in VQAcited by 5
- 2024BIV-Priv-Seg: Locating Private Content in Images Taken by People With Visual Impairmentscited by 5
- 2024Scene-Text Grounding for Text-Based Video Question Answeringcited by 5
- 2023Toward Unsupervised Realistic Visual Question Answeringcited by 4
- 2023Interpretable Visual Question Answering via Reasoning Supervisioncited by 4
- 2026Instance-Aligned Captions for Explainable Video Anomaly Detectioncited by 4
- 2023LOIS: Looking Out of Instance Semantics for Visual Question Answeringcited by 3
- 2026VISTAQA: Benchmarking Joint Visual Question Answering and Pixel-Level Evidencecited by 3
- 2023Adaptive loose optimization for robust question answeringcited by 2
- 2023A Survey on Interpretable Cross-modal Reasoningcited by 2
- 2024Multimodal Rationales for Explainable Visual Question Answeringcited by 2
- 2024NaVIP: An Image-Centric Indoor Navigation Solution for Visually Impaired Peoplecited by 2
- 2025Situat3DChange: Situated 3D Change Understanding Dataset for Multimodal Large Language Modelcited by 2
- 2022Tell Me the Evidence? Dual Visual-Linguistic Interaction for Answer Groundingcited by 1
- 2024SimpsonsVQA: Enhancing Inquiry-Based Learning with a Tailored Datasetcited by 1
- 2023Knowledge-Based Counterfactual Queries for Visual Question Answeringcited by 0
- 2025A Framework for Generating Artificial Datasets to Validate Absolute and Relative Position Conceptscited by 0
- 2025Hierarchical Instance Tracking to Balance Privacy Preservation with Accessible Informationcited by 0
- 2026Explicit Logic Channel for Validation and Enhancement of MLLMs on Zero-Shot Taskscited by 0
- 2026Counterfactual Reasoning for Fine-Grained Evidence Disentanglement in VideoQAcited by 0
- 2026DDVT: Dynamic Dual-level Vision Transformer Fusion Network for Answer Grounding in Visual Question Answeringcited by 0
- 2026ENCORE: Entropy-Guided Cropping and Attention Regularization for Robust Vision--Language Understandingcited by 0
- 2026DocAttriBench: Benchmarking Answer Grounding in Document Visual Question Answeringcited by 0
- 2026DocAttriBench: Benchmarking Answer Grounding in Document Visual Question Answering17 Sep 2026
- 2026ENCORE: Entropy-Guided Cropping and Attention Regularization for Robust Vision--Language Understanding24 Aug 2026
- 2026DDVT: Dynamic Dual-level Vision Transformer Fusion Network for Answer Grounding in Visual Question Answering27 Jul 2026
- 2026Counterfactual Reasoning for Fine-Grained Evidence Disentanglement in VideoQA8 Jun 2026
- 2026VISTAQA: Benchmarking Joint Visual Question Answering and Pixel-Level Evidence20 May 2026
- 2026Explicit Logic Channel for Validation and Enhancement of MLLMs on Zero-Shot Tasks12 Mar 2026
- 2026Instance-Aligned Captions for Explainable Video Anomaly Detection13 Jan 2026
- 2025Hierarchical Instance Tracking to Balance Privacy Preservation with Accessible Information10 Dec 2025
- 2025Situat3DChange: Situated 3D Change Understanding Dataset for Multimodal Large Language Model13 Oct 2025
- 2025A Framework for Generating Artificial Datasets to Validate Absolute and Relative Position Concepts17 Sep 2025
- 2025Towards Understanding Visual Grounding in Visual Language Models12 Sep 2025
- 2025RefChartQA: Grounding Visual Answer on Chart Images through Instruction Tuning29 Mar 2025
35 more
- 2025Survey of Adversarial Robustness in Multimodal Large Language Models18 Mar 2025
- 2025Acknowledging Focus Ambiguity in Visual Questions4 Jan 2025
- 2024Toward Visual Grounding: A Survey28 Dec 2024
- 2024Right this way: Can VLMs Guide Us to See More to Answer Questions?1 Nov 2024
- 2024SimpsonsVQA: Enhancing Inquiry-Based Learning with a Tailored Dataset30 Oct 2024
- 2024MC-Bench: A Benchmark for Multi-Context Visual Grounding in the Era of MLLMs16 Oct 2024
- 2024NaVIP: An Image-Centric Indoor Navigation Solution for Visually Impaired People8 Oct 2024
- 2024Scene-Text Grounding for Text-Based Video Question Answering22 Sep 2024
- 2024BIV-Priv-Seg: Locating Private Content in Images Taken by People With Visual Impairments25 Jul 2024
- 2024A Survey of Attacks on Large Vision-Language Models: Resources, Advances, and Future Trends10 Jul 2024
- 2024On the Role of Visual Grounding in VQA26 Jun 2024
- 2024OmniCount: Multi-label Object Counting with Semantic-Geometric Priors8 Mar 2024
- 2024GROUNDHOG: Grounding Large Language Models to Holistic Segmentation26 Feb 2024
- 2024Multimodal Rationales for Explainable Visual Question Answering6 Feb 2024
- 2023From Image to Language: A Critical Analysis of Visual Question Answering (VQA) Approaches, Challenges, and Opportunities1 Nov 2023
- 2023Toloka Visual Question Answering Benchmark28 Sep 2023
- 2023Sentence Attention Blocks for Answer Grounding20 Sep 2023
- 2023Interpretable Visual Question Answering via Reasoning Supervision7 Sep 2023
- 2023A Survey on Interpretable Cross-modal Reasoning5 Sep 2023
- 2023Can I Trust Your Answer? Visually Grounded Video Question Answering4 Sep 2023
- 2023VQA Therapy: Exploring Answer Differences by Visually Grounding Answers21 Aug 2023
- 2023An Outlook into the Future of Egocentric Vision14 Aug 2023
- 2023LOIS: Looking Out of Instance Semantics for Visual Question Answering26 Jul 2023
- 2023Dealing with Semantic Underspecification in Multimodal NLP8 Jun 2023
- 2023Adaptive loose optimization for robust question answering6 May 2023
- 2023Quality-agnostic Image Captioning to Safely Assist People with Vision Impairment28 Apr 2023
- 2023Logical Implications for Visual Question Answering Consistency16 Mar 2023
- 2023Toward Unsupervised Realistic Visual Question Answering9 Mar 2023
- 2023Knowledge-Based Counterfactual Queries for Visual Question Answering5 Mar 2023
- 2023Salient Object Detection for Images Taken by People With Vision Impairments12 Jan 2023
- 2022Pix2Struct: Screenshot Parsing as Pretraining for Visual Language Understanding7 Oct 2022
- 2022EPIC-KITCHENS VISOR Benchmark: VIdeo Segmentations and Object Relations26 Sep 2022
- 2022VizWiz-FewShot: Locating Objects in Images Taken by People With Visual Impairments24 Jul 2022
- 2022Tell Me the Evidence? Dual Visual-Linguistic Interaction for Answer Grounding21 Jun 2022
- 2022Unified-IO: A Unified Model for Vision, Language, and Multi-Modal Tasks17 Jun 2022
- 2024A Survey of Attacks on Large Vision-Language Models: Resources, Advances, and Future Trendscited by 82
- 2024GROUNDHOG: Grounding Large Language Models to Holistic Segmentationcited by 70
- 2024MC-Bench: A Benchmark for Multi-Context Visual Grounding in the Era of MLLMscited by 38
- 2025Survey of Adversarial Robustness in Multimodal Large Language Modelscited by 14
- 2025RefChartQA: Grounding Visual Answer on Chart Images through Instruction Tuningcited by 10
- 2025Towards Understanding Visual Grounding in Visual Language Modelscited by 9
- 2025Situat3DChange: Situated 3D Change Understanding Dataset for Multimodal Large Language Modelcited by 2
- 2026Explicit Logic Channel for Validation and Enhancement of MLLMs on Zero-Shot Taskscited by 0
- 2026ENCORE: Entropy-Guided Cropping and Attention Regularization for Robust Vision--Language Understandingcited by 0
- 2026ENCORE: Entropy-Guided Cropping and Attention Regularization for Robust Vision--Language Understanding24 Aug 2026
- 2026Explicit Logic Channel for Validation and Enhancement of MLLMs on Zero-Shot Tasks12 Mar 2026
- 2025Situat3DChange: Situated 3D Change Understanding Dataset for Multimodal Large Language Model13 Oct 2025
- 2025Towards Understanding Visual Grounding in Visual Language Models12 Sep 2025
- 2025RefChartQA: Grounding Visual Answer on Chart Images through Instruction Tuning29 Mar 2025
- 2025Survey of Adversarial Robustness in Multimodal Large Language Models18 Mar 2025
- 2024MC-Bench: A Benchmark for Multi-Context Visual Grounding in the Era of MLLMs16 Oct 2024
- 2024A Survey of Attacks on Large Vision-Language Models: Resources, Advances, and Future Trends10 Jul 2024
- 2024GROUNDHOG: Grounding Large Language Models to Holistic Segmentation26 Feb 2024
- 2022Unified-IO: A Unified Model for Vision, Language, and Multi-Modal Taskscited by 420
- 2022Pix2Struct: Screenshot Parsing as Pretraining for Visual Language Understandingcited by 360
- 2022EPIC-KITCHENS VISOR Benchmark: VIdeo Segmentations and Object Relationscited by 139
- 2023Can I Trust Your Answer? Visually Grounded Video Question Answeringcited by 133
- 2023An Outlook into the Future of Egocentric Visioncited by 84
- 2024Toward Visual Grounding: A Surveycited by 52
- 2023From Image to Language: A Critical Analysis of Visual Question Answering (VQA) Approaches, Challenges, and Opportunitiescited by 24
- 2024OmniCount: Multi-label Object Counting with Semantic-Geometric Priorscited by 16
- 2023VQA Therapy: Exploring Answer Differences by Visually Grounding Answerscited by 14
- 2022VizWiz-FewShot: Locating Objects in Images Taken by People With Visual Impairmentscited by 11
- 2023Toloka Visual Question Answering Benchmarkcited by 10
- 2024Right this way: Can VLMs Guide Us to See More to Answer Questions?cited by 10
12 most cited of 36 All 36 (JSON)
- 2026DocAttriBench: Benchmarking Answer Grounding in Document Visual Question Answering17 Sep 2026
- 2026DDVT: Dynamic Dual-level Vision Transformer Fusion Network for Answer Grounding in Visual Question Answering27 Jul 2026
- 2026Counterfactual Reasoning for Fine-Grained Evidence Disentanglement in VideoQA8 Jun 2026
- 2026Instance-Aligned Captions for Explainable Video Anomaly Detection13 Jan 2026
- 2025Hierarchical Instance Tracking to Balance Privacy Preservation with Accessible Information10 Dec 2025
- 2025A Framework for Generating Artificial Datasets to Validate Absolute and Relative Position Concepts17 Sep 2025
- 2025Acknowledging Focus Ambiguity in Visual Questions4 Jan 2025
- 2024Toward Visual Grounding: A Survey28 Dec 2024
- 2024Right this way: Can VLMs Guide Us to See More to Answer Questions?1 Nov 2024
- 2024SimpsonsVQA: Enhancing Inquiry-Based Learning with a Tailored Dataset30 Oct 2024
- 2024NaVIP: An Image-Centric Indoor Navigation Solution for Visually Impaired People8 Oct 2024
- 2024Scene-Text Grounding for Text-Based Video Question Answering22 Sep 2024
12 newest of 36 All 36 (JSON)