# How Multimodal Large Language Models Support Access to Visual Information: A Diary Study With Blind and Low Vision People

arXiv: https://arxiv.org/abs/2602.13469 · HTML: https://mlsanity.com/paper/2602.13469/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 36 of its 86 references that are in the map.
- First posted on arXiv 13 Feb 2026.

## Placed in

- Multimodal LLMs / vision-language models / [VLM benchmarks and visual reasoning](https://mlsanity.com/mllm/vlm-benchmarks-and-visual-reasoning/)

## Builds on (most cited first, 30 of 36)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [CIDEr: Consensus-based Image Description Evaluation](https://mlsanity.com/paper/1411.5726/) (2014)
- [ShareGPT4V: Improving Large Multi-Modal Models with Better Captions](https://mlsanity.com/paper/2311.12793/) (2023)
- [SPICE: Semantic Propositional Image Caption Evaluation](https://mlsanity.com/paper/1607.08822/) (2016)
- [The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)](https://mlsanity.com/paper/2309.17421/) (2023)
- [Why Language Models Hallucinate](https://mlsanity.com/paper/2509.04664/) (2025)
- [GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks](https://mlsanity.com/paper/2510.04374/) (2025)
- [Improve Vision Language Model Chain-of-thought Reasoning](https://mlsanity.com/paper/2410.16198/) (2024)
- [The rising costs of training frontier AI models](https://mlsanity.com/paper/2405.21015/) (2024)
- [CapArena: Benchmarking and Analyzing Detailed Image Captioning in the LLM Era](https://mlsanity.com/paper/2503.12329/) (2025)
- [Believing Anthropomorphism: Examining the Role of Anthropomorphic Cues on Trust in Large Language Models](https://mlsanity.com/paper/2405.06079/) (2024)
- [Be Friendly, Not Friends: How LLM Sycophancy Shapes User Trust](https://mlsanity.com/paper/2502.10844/) (2025)
- [Do Human Rationales Improve Machine Explanations?](https://mlsanity.com/paper/1905.13714/) (2019)
- [A Review of Multi-Modal Large Language and Vision Models](https://mlsanity.com/paper/2404.01322/) (2024)
- [WorldScribe: Towards Context-Aware Live Visual Descriptions](https://mlsanity.com/paper/2408.06627/) (2024)
- [Investigating Use Cases of AI-Powered Scene Description Applications for Blind and Low Vision People](https://mlsanity.com/paper/2403.15604/) (2024)
- [Personalizing Multimodal Large Language Models for Image Captioning: An Experimental Analysis](https://mlsanity.com/paper/2412.03665/) (2024)
- [Toward Robust Hyper-Detailed Image Captioning: A Multiagent Approach and Dual Evaluation Metrics for Factuality and Coverage](https://mlsanity.com/paper/2412.15484/) (2024)
- [Long-Form Answers to Visual Questions from Blind and Low Vision People](https://mlsanity.com/paper/2408.06303/) (2024)
- [Misfitting With AI: How Blind People Verify and Contest AI Errors](https://mlsanity.com/paper/2408.06546/) (2024)
- [Beyond Visual Perception: Insights from Smartphone Interaction of Visually Impaired Users with Large Multimodal Models](https://mlsanity.com/paper/2502.16098/) (2025)
- [Emerging Practices for Large Multimodal Model (LMM) Assistance for People with Visual Impairments: Implications for Design](https://mlsanity.com/paper/2407.08882/) (2024)
- [Fully Authentic Visual Question Answering Dataset from Online Communities](https://mlsanity.com/paper/2311.15562/) (2023)
- [A Contextual Inquiry of People with Vision Impairments in Cooking](https://mlsanity.com/paper/2402.15108/) (2024)
- [A Recipe for Success? Exploring Strategies for Improving Non-Visual Access to Cooking Instructions](https://mlsanity.com/paper/2407.19065/) (2024)
- [Context-Aware Image Descriptions for Web Accessibility](https://mlsanity.com/paper/2409.03054/) (2024)
- [Conversations for Vision: Remote Sighted Assistants Helping People with Visual Impairments](https://mlsanity.com/paper/1812.00148/) (2018)
- [Generating Rationales in Visual Question Answering](https://mlsanity.com/paper/2004.02032/) (2020)
- [Data Representativeness in Accessibility Datasets: A Meta-Analysis](https://mlsanity.com/paper/2207.08037/) (2022)
- [ImageAssist: Tools for Enhancing Touchscreen-Based Image Exploration Systems for Blind and Low Vision Users](https://mlsanity.com/paper/2302.09124/) (2023)

## Built on by (most cited first, 3 of 3)

- [Querying Multimodal Scientific Papers with AI: Practices and Preferences Across Blind, Low-Vision, and Sighted Scientists](https://mlsanity.com/paper/2607.18514/) (2026)
- [Touvigation: Embodied Adaptive Object Acquisition for Blind and Low-Vision Users in Unfamiliar Indoor Environments](https://mlsanity.com/paper/2609.21828/) (2026)
- [Devils in Question Relay: Source-Conditioned Relay Steering to Mitigate Hallucinations in Audio-visual Large Language Models](https://mlsanity.com/paper/2609.37568/) (2026)

## Abstract (arXiv)

Multimodal large language models (MLLMs) are changing how Blind and Low Vision (BLV) people access visual information. Unlike traditional visual interpretation tools that only provide descriptions, MLLM-enabled applications offer conversational assistance, where users can ask questions to obtain goal-relevant details. However, evidence about their performance in the real-world and implications for BLV people's daily lives remains limited. To address this, we conducted a two-week diary study, where we captured 20 BLV participants' use of an MLLM-enabled visual interpretation application. Although participants rated the visual interpretations of the application as "trustworthy" (mean=3.76 out of 5, max=extremely trustworthy) and "somewhat satisfying" (mean=4.13 out of 5, max=very satisfying), the AI often produced incorrect answers (22.2%) or abstained (10.8%) from responding to users' requests. Our findings show that while MLLMs can improve visual interpretations' descriptive accuracy, supporting everyday use also depends on the "visual assistant" skill: behaviors for providing goal-directed, reliable assistance. We conclude by proposing the "visual assistant" skill and guidelines to help MLLM-enabled visual interpretation applications better support BLV people's access to visual information.