# Making Videos Accessible for Blind and Low Vision Users Using a Multimodal Agent Video Player

arXiv: https://arxiv.org/abs/2602.04104 · HTML: https://mlsanity.com/paper/2602.04104/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use.
- Builds on 24 of its 73 references that are in the map.
- First posted on arXiv 4 Feb 2026.

## Placed in

- LLM agents and tool use / [Multimodal and embodied agents](https://mlsanity.com/agents/multimodal-and-embodied-agents/)

## Builds on (most cited first, 24 of 24)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Ethical and social risks of harm from Language Models](https://mlsanity.com/paper/2112.04359/) (2021)
- [VideoAgent: Long-form Video Understanding with Large Language Model as Agent](https://mlsanity.com/paper/2403.10517/) (2024)
- [VideoAgent: A Memory-augmented Multimodal Agent for Video Understanding](https://mlsanity.com/paper/2403.11481/) (2024)
- [The Ethics of Advanced AI Assistants](https://mlsanity.com/paper/2404.16244/) (2024)
- [Investigating Use Cases of AI-Powered Scene Description Applications for Blind and Low Vision People](https://mlsanity.com/paper/2403.15604/) (2024)
- [Rescribe: Authoring and Automatically Editing Audio Descriptions](https://mlsanity.com/paper/2010.03667/) (2020)
- [AVscript: Accessible Video Editing with Audio-Visual Scripts](https://mlsanity.com/paper/2302.14117/) (2023)
- [A Systematic Review and Thematic Analysis of Community-Collaborative Approaches to Computing Research](https://mlsanity.com/paper/2207.04171/) (2022)
- [Making Short-Form Videos Accessible with Hierarchical Video Summaries](https://mlsanity.com/paper/2402.10382/) (2024)
- [Vid2Coach: Transforming How-To Videos into Task Assistants](https://mlsanity.com/paper/2506.00717/) (2025)
- ["It's Kind of Context Dependent": Understanding Blind and Low Vision People's Video Accessibility Preferences Across Viewing Scenarios](https://mlsanity.com/paper/2403.10792/) (2024)
- [Towards Understanding the Use of MLLM-Enabled Applications for Visual Interpretation by Blind and Low Vision People](https://mlsanity.com/paper/2503.05899/) (2025)
- [Say It All: Feedback for Improving Non-Visual Presentation Accessibility](https://mlsanity.com/paper/2103.14491/) (2021)
- [CUI@CHI 2024: Building Trust in CUIs-From Design to Deployment](https://mlsanity.com/paper/2401.13970/) (2024)
- [SPICA: Interactive Video Content Exploration through Augmented Audio Descriptions for Blind or Low-Vision Viewers](https://mlsanity.com/paper/2402.07300/) (2024)
- [Audio Description Generation in the Era of LLMs and VLMs: A Review of Transferable Generative AI Technologies](https://mlsanity.com/paper/2410.08860/) (2024)
- [Describe Now: User-Driven Audio Description for Blind and Low Vision Individuals](https://mlsanity.com/paper/2411.11835/) (2024)
- [StreetReaderAI: Making Street View Accessible Using Context-Aware Multimodal AI](https://mlsanity.com/paper/2508.08524/) (2025)
- [Audio Description Customization](https://mlsanity.com/paper/2408.11406/) (2024)
- [Exploring Community-Driven Descriptions for Making Livestreams Accessible](https://mlsanity.com/paper/2310.07057/) (2023)
- [CoSight: Exploring Viewer Contributions to Online Video Accessibility Through Descriptive Commenting](https://mlsanity.com/paper/2508.08582/) (2025)

## Abstract (arXiv)

Video content remains largely inaccessible to blind and low-vision (BLV) users. To address this, we introduce a prototype that leverages a multimodal agent - powered by a novel conversational architecture using a multimodal large language model (MLLM) - to provide BLV users with an interactive, accessible video experience. This Multimodal Agent Video Player (MAVP) demonstrates that an interactive accessibility mode can be added to a video through multilayered prompt orchestration. We describe a user-centered design process involving 18 sessions with BLV users that showed that BLV users do not just want accessibility features, but desire independence and personal agency over the viewing experience. We conducted a qualitative study with an additional 8 BLV participants; in this, we saw that the MAVP's conversational dialogue offers BLV users a sense of personal agency, fostering collaboration and trust. Even in the case of hallucinations, it is meta-conversational dialogues about AI's limitations that can repair trust.