# Towards Understanding the Use of MLLM-Enabled Applications for Visual Interpretation by Blind and Low Vision People

arXiv: https://arxiv.org/abs/2503.05899 · HTML: https://mlsanity.com/paper/2503.05899/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 24 references that are in the map.
- First posted on arXiv 7 Mar 2025.

## Builds on (most cited first, 8 of 8)

- [The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)](https://mlsanity.com/paper/2309.17421/) (2023)
- [A Review of Multi-Modal Large Language and Vision Models](https://mlsanity.com/paper/2404.01322/) (2024)
- [Investigating Use Cases of AI-Powered Scene Description Applications for Blind and Low Vision People](https://mlsanity.com/paper/2403.15604/) (2024)
- [Long-Form Answers to Visual Questions from Blind and Low Vision People](https://mlsanity.com/paper/2408.06303/) (2024)
- [Misfitting With AI: How Blind People Verify and Contest AI Errors](https://mlsanity.com/paper/2408.06546/) (2024)
- [Fully Authentic Visual Question Answering Dataset from Online Communities](https://mlsanity.com/paper/2311.15562/) (2023)
- [Conversations for Vision: Remote Sighted Assistants Helping People with Visual Impairments](https://mlsanity.com/paper/1812.00148/) (2018)
- ["We are at the mercy of others' opinion": Supporting Blind People in Recreational Window Shopping with AI-infused Technology](https://mlsanity.com/paper/2405.06611/) (2024)

## Built on by (most cited first, 11 of 11)

- ["Before, I Asked My Mom, Now I Ask ChatGPT": Visual Privacy Management with Generative AI for Blind and Low-Vision People](https://mlsanity.com/paper/2507.00286/) (2025)
- [Probing the Gaps in ChatGPT Live Video Chat for Real-World Assistance for People who are Blind or Visually Impaired](https://mlsanity.com/paper/2508.03651/) (2025)
- [Say It My Way: Exploring Control in Conversational Visual Question Answering with Blind Users](https://mlsanity.com/paper/2602.16930/) (2026)
- ["Accessibility people, you go work on that thing of yours over there": Addressing Disability Inclusion in AI Product Organizations](https://mlsanity.com/paper/2508.16607/) (2025)
- [BLaVe-CoT: Consistency-Aware Visual Question Answering for Blind and Low Vision Users](https://mlsanity.com/paper/2509.06010/) (2025)
- [Finding the Signal in the Noise: An Exploratory Study on Assessing the Effectiveness of AI and Accessibility Forums for Blind Users' Support Needs](https://mlsanity.com/paper/2602.18623/) (2026)
- [Guiding Multimodal Large Language Models with Blind and Low Vision People Visual Questions for Proactive Visual Interpretations](https://mlsanity.com/paper/2510.01576/) (2025)
- [Making Videos Accessible for Blind and Low Vision Users Using a Multimodal Agent Video Player](https://mlsanity.com/paper/2602.04104/) (2026)
- [Lost in Instructions: Study of Blind Users' Experiences with DIY Manuals and AI-Rewritten Instructions for Assembly, Operation, and Troubleshooting of Tangible Products](https://mlsanity.com/paper/2602.18630/) (2026)
- [Understanding the Use of a Large Language Model-Powered Guide to Make Virtual Reality Accessible for Blind and Low Vision People](https://mlsanity.com/paper/2603.09964/) (2026)
- [Touvigation: Embodied Adaptive Object Acquisition for Blind and Low-Vision Users in Unfamiliar Indoor Environments](https://mlsanity.com/paper/2609.21828/) (2026)

## Abstract (arXiv)

Blind and Low Vision (BLV) people have adopted AI-powered visual interpretation applications to address their daily needs. While these applications have been helpful, prior work has found that users remain unsatisfied by their frequent errors. Recently, multimodal large language models (MLLMs) have been integrated into visual interpretation applications, and they show promise for more descriptive visual interpretations. However, it is still unknown how this advancement has changed people's use of these applications. To address this gap, we conducted a two-week diary study in which 20 BLV people used an MLLM-enabled visual interpretation application we developed, and we collected 553 entries. In this paper, we report a preliminary analysis of 60 diary entries from 6 participants. We found that participants considered the application's visual interpretations trustworthy (mean 3.75 out of 5) and satisfying (mean 4.15 out of 5). Moreover, participants trusted our application in high-stakes scenarios, such as receiving medical dosage advice. We discuss our plan to complete our analysis to inform the design of future MLLM-enabled visual interpretation systems.