# What You See is What You Ask: Evaluating Audio Descriptions

arXiv: https://arxiv.org/abs/2510.00808 · HTML: https://mlsanity.com/paper/2510.00808/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 34 references that are in the map.
- First posted on arXiv 1 Oct 2025.

## Builds on (most cited first, 21 of 21)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [CIDEr: Consensus-based Image Description Evaluation](https://mlsanity.com/paper/1411.5726/) (2014)
- [TVQA: Localized, Compositional Video Question Answering](https://mlsanity.com/paper/1809.01696/) (2018)
- [WhisperX: Time-Accurate Speech Transcription of Long-Form Audio](https://mlsanity.com/paper/2303.00747/) (2023)
- [MovieQA: Understanding Stories in Movies through Question-Answering](https://mlsanity.com/paper/1512.02902/) (2015)
- [Movie Description](https://mlsanity.com/paper/1605.03705/) (2016)
- [Condensed Movies: Story Based Retrieval with Contextual Embeddings](https://mlsanity.com/paper/2005.04208/) (2020)
- [AutoAD II: The Sequel -- Who, When, and What in Movie Audio Description](https://mlsanity.com/paper/2310.06838/) (2023)
- [MM-Narrator: Narrating Long-form Videos with Multimodal In-Context Learning](https://mlsanity.com/paper/2311.17435/) (2023)
- [AutoAD III: The Prequel -- Back to the Pixels](https://mlsanity.com/paper/2404.14412/) (2024)
- [AutoAD-Zero: A Training-Free Framework for Zero-Shot Audio Description](https://mlsanity.com/paper/2407.15850/) (2024)
- [Rescribe: Authoring and Automatically Editing Audio Descriptions](https://mlsanity.com/paper/2010.03667/) (2020)
- [LLM-AD: Large Language Model based Audio Description System](https://mlsanity.com/paper/2405.00983/) (2024)
- [Identity-Aware Multi-Sentence Video Description](https://mlsanity.com/paper/2008.09791/) (2020)
- [Shot-by-Shot: Film-Grammar-Aware Training-Free Audio Description Generation](https://mlsanity.com/paper/2504.01020/) (2025)
- [DistinctAD: Distinctive Audio Description Generation in Contexts](https://mlsanity.com/paper/2411.18180/) (2024)
- [MICap: A Unified Model for Identity-aware Movie Descriptions](https://mlsanity.com/paper/2405.11483/) (2024)
- [Audio Description Generation in the Era of LLMs and VLMs: A Review of Transferable Generative AI Technologies](https://mlsanity.com/paper/2410.08860/) (2024)
- [DANTE-AD: Dual-Vision Attention Network for Long-Term Audio Description](https://mlsanity.com/paper/2503.24096/) (2025)
- [FocusedAD: Character-centric Movie Audio Description](https://mlsanity.com/paper/2504.12157/) (2025)

## Built on by (most cited first, 8 of 8)

- [More than a Moment: Towards Coherent Sequences of Audio Descriptions](https://mlsanity.com/paper/2510.25440/) (2025)
- [REFRAMED: Towards Realistic Audio Description Generation for Movies](https://mlsanity.com/paper/2608.09765/) (2026)
- [From Visual Cues to Spoken Narration: Rethinking Audio Description](https://mlsanity.com/paper/2609.01725/) (2026)
- [Tracing Audio Grounding and Answer Selection in Audio LLMs](https://mlsanity.com/paper/2609.04637/) (2026)
- [Andha-Dhun: A First Look at Audio Descriptions in Hindi](https://mlsanity.com/paper/2607.06457/) (2026)
- [StoryTeller: Training-Free Narrative Grounding for Long-Form Audio Description](https://mlsanity.com/paper/2607.11798/) (2026)
- [StrAD: A Streaming Method and Benchmark for Audio Description Generation for Long-form Videos](https://mlsanity.com/paper/2608.12549/) (2026)
- [AD4EDU: Benchmarking University Lecture Audio Description across Accessibility Standards, Metrics, and Learner Preferences](https://mlsanity.com/paper/2610.17042/) (2026)

## Abstract (arXiv)

Audio descriptions (ADs) narrate important visual details in movies, enabling Blind and Low Vision (BLV) users to understand narratives and appreciate visual details. Existing works in automatic AD generation mostly focus on few-second trimmed clips, and evaluate them by comparing against a single ground-truth reference AD. However, writing ADs is inherently subjective. Through alignment and analysis of two independent AD tracks for the same movies, we quantify the subjectivity in when and whether to describe, and what and how to highlight. Thus, we show that working with trimmed clips is inadequate. We propose ADQA, a QA benchmark that evaluates ADs at the level of few-minute long, coherent video segments, testing whether they would help BLV users understand the story and appreciate visual details. ADQA features visual appreciation (VA) questions about visual facts and narrative understanding (NU) questions based on the plot. Through ADQA, we show that current AD generation methods lag far behind human-authored ADs. We conclude with several recommendations for future work and introduce a public leaderboard for benchmarking.