mlsanityData to 13 Oct 2026

Long-Form Answers to Visual Questions from Blind and Low Vision People

arXiv 2408.06303 ↗First posted 12 Aug 2024cs.CL cs.CVcited by 18

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning

Abstract (arXiv)

Vision language models can now generate long-form answers to questions about images - long-form visual question answers (LFVQA). We contribute VizWiz-LF, a dataset of long-form answers to visual questions posed by blind and low vision (BLV) users. VizWiz-LF contains 4.2k long-form answers to 600 visual questions, collected from human expert describers and six VQA models. We develop and annotate functional roles of sentences of LFVQA and demonstrate that long-form answers contain information beyond the question answer such as explanations and suggestions. We further conduct automatic and human evaluations with BLV and sighted people to evaluate long-form answers. BLV people perceive both human-written and generated long-form answers to be plausible, but generated answers often hallucinate incorrect visual details, especially for unanswerable visual questions (e.g., blurry or irrelevant images). To reduce hallucinations, we evaluate the ability of VQA models to abstain from answering unanswerable questions across multiple prompting strategies.

Key facts. Cited by 18 papers in the map (data through 13 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 55 of its 82 references that are in the map. First posted on arXiv 12 Aug 2024.

Builds on

55 of 82 references.

Closest ancestors · 3

Same direction, newest first

  1. 2023An Evaluation of GPT-4V and Gemini in Online VQA17 Dec
  2. 2023PaLI-3 Vision Language Models: Smaller, Faster, Stronger13 Oct
  3. 2022Flamingo: a Visual Language Model for Few-Shot Learning29 Apr

Other references · 52

  1. 2023GPT-4 Technical Reportcited by 18,278
  2. 2023Visual Instruction Tuningcited by 7,827
  3. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,103
  4. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,657
  5. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,480
  6. 2022Scaling Instruction-Finetuned Language Modelscited by 3,007
  7. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,864
  8. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,649
  9. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,506
  10. 2015VQA: Visual Question Answeringcited by 2,495
  11. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,057
  12. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,052
38 more
  1. 2022Language Models (Mostly) Know What They Knowcited by 1,823
  2. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,736
  3. 2023Evaluating Object Hallucination in Large Vision-Language Modelscited by 1,709
  4. 2023Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMscited by 960
  5. 2018VizWiz Grand Challenge: Answering Visual Questions from Blind Peoplecited by 852
  6. 2022EVA: Exploring the Limits of Masked Visual Representation Learning at Scalecited by 689
  7. 2023DataComp: In search of the next generation of multimodal datasetscited by 639
  8. 2019ELI5: Long Form Question Answeringcited by 571
  9. 2015Visual7W: Grounded Question Answering in Imagescited by 316
  10. 2022Teaching language models to support answers with verified quotescited by 291
  11. 2022Generating Sequences by Learning to Self-Correctcited by 273
  12. 2023Universal Self-Consistency for Large Language Model Generationcited by 200
  13. 2016FVQA: Fact-based Visual Question Answeringcited by 181
  14. 2024Don't Hallucinate, Abstain: Identifying LLM Knowledge Gaps via Multi-LLM Collaborationcited by 179
  15. 2021Hurdles to Progress in Long-form Question Answeringcited by 166
  16. 2023Holistic Analysis of Hallucination in GPT-4V(ision): Bias and Interference Challengescited by 131
  17. 2016Visual Question Answering: A Survey of Methods and Datasetscited by 123
  18. 2023A Critical Evaluation of Evaluations for Long-form Question Answeringcited by 111
  19. 2021Decontextualization: Making Sentences Stand-Alonecited by 101
  20. 2023Evaluation and Analysis of Hallucination in Large Vision-Language Modelscited by 99
  21. 2022Reliable Visual Question Answering: Abstain Rather Than Answer Incorrectlycited by 86
  22. 2023Context-faithful Prompting for Large Language Modelscited by 79
  23. 2019Investigating Evaluation of Open-Domain Dialogue Systems With Human Generated Multiple Referencescited by 48
  24. 2023Improving Automatic VQA Evaluation Using Large Language Modelscited by 37
  25. 2024Calibrating Long-form Generations from Large Language Modelscited by 32
  26. 2020Beyond VQA: Generating Multi-word Answer and Rationale to Visual Questionscited by 19
  27. 2022How Do We Answer Complex Questions: Discourse Structure of Long-form Answerscited by 19
  28. 2023AVscript: Accessible Video Editing with Audio-Visual Scriptscited by 19
  29. 2022Context Matters for Image Descriptions for Accessibility: Challenges for Referenceless Evaluation Metricscited by 18
  30. 2023GenAssist: Making Image Generation Accessiblecited by 18
  31. 2019Quality Estimation for Image Captions Based on Large-scale Human Evaluationscited by 17
  32. 2020Vision Skills Needed to Answer Visual Questionscited by 17
  33. 2023VQA Therapy: Exploring Answer Differences by Visually Grounding Answerscited by 14
  34. 2023Which is better? Exploring Prompting Strategy For LLM-based Metricscited by 14
  35. 2024Making Short-Form Videos Accessible with Hierarchical Video Summariescited by 14
  36. 2023Fully Authentic Visual Question Answering Dataset from Online Communitiescited by 9
  37. 2023Context-VQA: Towards Context-Aware and Purposeful Visual Question Answeringcited by 6
  38. 2020Determining Question-Answer Plausibility in Crowdsourced Datasets Using Multi-Task Learningcited by 5

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2025: 12; 2026: 6.

18 papers.

Sort:

By area:

  1. 2025MIRAGE: A Benchmark for Multimodal Information-Seeking and Reasoning in Agricultural Expert-Guided Conversationscited by 13
  2. 2025Vid2Coach: Transforming How-To Videos into Task Assistantscited by 12
  3. 2025Towards Understanding the Use of MLLM-Enabled Applications for Visual Interpretation by Blind and Low Vision Peoplecited by 11
  4. 2025Evaluating Multimodal Language Models as Visual Assistants for Visually Impaired Userscited by 11
  5. 2025Acknowledging Focus Ambiguity in Visual Questionscited by 7
  6. 2025Surfacing Variations to Calibrate Perceived Reliability of MLLM-generated Image Descriptionscited by 6
  7. 2025"It's trained by non-disabled people": Evaluating How Image Quality Affects Product Captioning with Vision-Language Modelscited by 3
  8. 2025MAVIS: A Benchmark for Multimodal Source Attribution in Long-form Visual Question Answeringcited by 3
  9. 2026How Multimodal Large Language Models Support Access to Visual Information: A Diary Study With Blind and Low Vision Peoplecited by 3
  10. 2026Say It My Way: Exploring Control in Conversational Visual Question Answering with Blind Userscited by 2
  11. 2026Semantic visually-guided acoustic highlighting with large vision-language modelscited by 1
  12. 2025EmoAssist: Emotional Assistant for Visual Impairment Communitycited by 0
6 more
  1. 2025Believing without Seeing: Quality Scores for Contextualizing Vision-Language Model Explanationscited by 0
  2. 2025Guiding Multimodal Large Language Models with Blind and Low Vision People Visual Questions for Proactive Visual Interpretationscited by 0
  3. 2025Hierarchical Instance Tracking to Balance Privacy Preservation with Accessible Informationcited by 0
  4. 2026MacVQA: Adaptive Memory Allocation and Global Noise Filtering for Continual Visual Question Answeringcited by 0
  5. 2026Cripping AI: Reimagining AI Through Lived Disability Experiencescited by 0
  6. 2026How Much Does It Cost to Answer My Question? Benchmarking Cloud VLM-based VQA Systemscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025BLaVe-CoT: Consistency-Aware Visual Question Answering for Blind and Low Vision Users
  2. 2024Right this way: Can VLMs Guide Us to See More to Answer Questions?
  3. 2024TUBench: Benchmarking Large Vision-Language Models on Trustworthiness with Unanswerable Questions
  4. 2025VisionTrap: Unanswerable Questions On Visual Data
  5. 2025Where do Large Vision-Language Models Look at when Answering Questions?
  6. 2025The Quest for Visual Understanding: A Journey Through the Evolution of Visual Question Answering
  7. 2024Q-Bench+: A Benchmark for Multi-modal Foundation Models on Low-level Vision from Single Images to Pairs
  8. 2025Visual question answering: from early developments to recent advances -- a survey