Making Videos Accessible for Blind and Low Vision Users Using a Multimodal Agent Video Player
On the map: LLM agents and tool use → Multimodal and embodied agents
Abstract (arXiv)
Video content remains largely inaccessible to blind and low-vision (BLV) users. To address this, we introduce a prototype that leverages a multimodal agent - powered by a novel conversational architecture using a multimodal large language model (MLLM) - to provide BLV users with an interactive, accessible video experience. This Multimodal Agent Video Player (MAVP) demonstrates that an interactive accessibility mode can be added to a video through multilayered prompt orchestration. We describe a user-centered design process involving 18 sessions with BLV users that showed that BLV users do not just want accessibility features, but desire independence and personal agency over the viewing experience. We conducted a qualitative study with an additional 8 BLV participants; in this, we saw that the MAVP's conversational dialogue offers BLV users a sense of personal agency, fostering collaboration and trust. Even in the case of hallucinations, it is meta-conversational dialogues about AI's limitations that can repair trust.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 24 of its 73 references that are in the map. First posted on arXiv 4 Feb 2026.
Builds on
24 of 73 references.
Closest ancestors · 2
Same direction, newest first
- 2024VideoAgent: A Memory-augmented Multimodal Agent for Video Understanding18 Mar
- 2024VideoAgent: Long-form Video Understanding with Large Language Model as Agent15 Mar
Other references · 22
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2023GPT-4 Technical Reportcited by 18,132
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
- 2021Ethical and social risks of harm from Language Modelscited by 957
- 2024The Ethics of Advanced AI Assistantscited by 103
- 2024Investigating Use Cases of AI-Powered Scene Description Applications for Blind and Low Vision Peoplecited by 24
- 2020Rescribe: Authoring and Automatically Editing Audio Descriptionscited by 20
- 2023AVscript: Accessible Video Editing with Audio-Visual Scriptscited by 19
- 2024Making Short-Form Videos Accessible with Hierarchical Video Summariescited by 14
- 2022A Systematic Review and Thematic Analysis of Community-Collaborative Approaches to Computing Researchcited by 13
- 2025Vid2Coach: Transforming How-To Videos into Task Assistantscited by 12
- 2025Towards Understanding the Use of MLLM-Enabled Applications for Visual Interpretation by Blind and Low Vision Peoplecited by 11
10 more
- 2024"It's Kind of Context Dependent": Understanding Blind and Low Vision People's Video Accessibility Preferences Across Viewing Scenarioscited by 10
- 2021Say It All: Feedback for Improving Non-Visual Presentation Accessibilitycited by 9
- 2024CUI@CHI 2024: Building Trust in CUIs-From Design to Deploymentcited by 6
- 2024SPICA: Interactive Video Content Exploration through Augmented Audio Descriptions for Blind or Low-Vision Viewerscited by 5
- 2025StreetReaderAI: Making Street View Accessible Using Context-Aware Multimodal AIcited by 5
- 2024Audio Description Customizationcited by 4
- 2024Audio Description Generation in the Era of LLMs and VLMs: A Review of Transferable Generative AI Technologiescited by 4
- 2024Describe Now: User-Driven Audio Description for Blind and Low Vision Individualscited by 4
- 2023Exploring Community-Driven Descriptions for Making Livestreams Accessiblecited by 1
- 2025CoSight: Exploring Viewer Contributions to Online Video Accessibility Through Descriptive Commentingcited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Explainable AI for Blind and Low-Vision Users: Navigating Trust, Modality, and Interpretability in the Agentic Era
- 2026How Multimodal Large Language Models Support Access to Visual Information: A Diary Study With Blind and Low Vision People
- 2026Bespoke Visual Assistance: What and How do Blind and Low-Vision People Create with Agentic Programming?
- 2025GamerAstra: Supporting 2D Non-Twitch Video Games for Blind and Low-Vision Players through a Multi-Agent Framework
- 2026ViDscribe: Multimodal AI for Customizing Audio Description and Question Answering in Online Videos
- 2025VideoA11y: Method and Dataset for Accessible Video Description
- 2026ADCanvas: Accessible and Conversational Audio Description Authoring for Blind and Low Vision Creators
- 2025Probing the Gaps in ChatGPT Live Video Chat for Real-World Assistance for People who are Blind or Visually Impaired