# Making Short-Form Videos Accessible with Hierarchical Video Summaries

arXiv: https://arxiv.org/abs/2402.10382 · HTML: https://mlsanity.com/paper/2402.10382/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Feb 2024.

## Built on by (most cited first, 14 of 14)

- [WorldScribe: Towards Context-Aware Live Visual Descriptions](https://mlsanity.com/paper/2408.06627/) (2024)
- [Long-Form Answers to Visual Questions from Blind and Low Vision People](https://mlsanity.com/paper/2408.06303/) (2024)
- [Vid2Coach: Transforming How-To Videos into Task Assistants](https://mlsanity.com/paper/2506.00717/) (2025)
- [VideoA11y: Method and Dataset for Accessible Video Description](https://mlsanity.com/paper/2502.20480/) (2025)
- [EditScribe: Non-Visual Image Editing with Natural Language Verification Loops](https://mlsanity.com/paper/2408.06632/) (2024)
- [Surfacing Variations to Calibrate Perceived Reliability of MLLM-generated Image Descriptions](https://mlsanity.com/paper/2507.15692/) (2025)
- ["It's trained by non-disabled people": Evaluating How Image Quality Affects Product Captioning with Vision-Language Models](https://mlsanity.com/paper/2511.08917/) (2025)
- [AltCanvas: A Tile-Based Image Editor with Generative AI for Blind or Visually Impaired People](https://mlsanity.com/paper/2408.10240/) (2024)
- [Toward Scalable Audio Description Quality Control: A Workflow for Evaluating Human and VLM Raters](https://mlsanity.com/paper/2602.01390/) (2026)
- [Designing Multi-Robot Ground Video Sensemaking with Public Safety Professionals](https://mlsanity.com/paper/2602.08882/) (2026)
- [GamerAstra: Supporting 2D Non-Twitch Video Games for Blind and Low-Vision Players through a Multi-Agent Framework](https://mlsanity.com/paper/2506.22937/) (2025)
- [Making Videos Accessible for Blind and Low Vision Users Using a Multimodal Agent Video Player](https://mlsanity.com/paper/2602.04104/) (2026)
- [Lost in Instructions: Study of Blind Users' Experiences with DIY Manuals and AI-Rewritten Instructions for Assembly, Operation, and Troubleshooting of Tangible Products](https://mlsanity.com/paper/2602.18630/) (2026)
- [How Do Novice Describers Edit AI Drafts? The Role of Draft Quality in Audio Description Authoring](https://mlsanity.com/paper/2605.05348/) (2026)

## Abstract (arXiv)

Short videos on platforms such as TikTok, Instagram Reels, and YouTube Shorts (i.e. short-form videos) have become a primary source of information and entertainment. Many short-form videos are inaccessible to blind and low vision (BLV) viewers due to their rapid visual changes, on-screen text, and music or meme-audio overlays. In our formative study, 7 BLV viewers who regularly watched short-form videos reported frequently skipping such inaccessible content. We present ShortScribe, a system that provides hierarchical visual summaries of short-form videos at three levels of detail to support BLV viewers in selecting and understanding short-form videos. ShortScribe allows BLV users to navigate between video descriptions based on their level of interest. To evaluate ShortScribe, we assessed description accuracy and conducted a user study with 10 BLV participants comparing ShortScribe to a baseline interface. When using ShortScribe, participants reported higher comprehension and provided more accurate summaries of video content.