StrAD: A Streaming Method and Benchmark for Audio Description Generation for Long-form Videos
Not on the map. Why?
Abstract (arXiv)
Visual content is the dominant medium of communication, yet without audio descriptions (ADs), it remains inaccessible to blind and low-vision people. ADs narrate context-relevant visual events during natural audio pauses. Manually creating ADs is expensive, limiting coverage to a small fraction of available content. Most existing automatic AD generation methods frame the task as video clip captioning, requiring ground-truth timestamps and additional context cues such as character databases. Current benchmarks reinforce this framing, consisting of short video segments paired with automatic or task-mismatched annotations. We introduce StrAD, a benchmark for long-form AD generation on full-length videos spanning diverse genres such as movies, documentaries, short films, performances, and video games. We reformulate AD generation as streaming dense video captioning. Our approach processes full-length videos with a sliding window, inserting ADs into existing transcripts without ground-truth timestamps, and supports both fine-tuned models and zero-shot prompting of vision-language models. On the segment-level task with given timestamps, our fine-tuned StrAD-FT sets the state of the art on CMD-AD with 36.3 CIDEr (+10.0 over Shot-by-shot), establishes a reference point on StrAD (51.0 CIDEr), and remains competitive on MAD-Eval at 24.9 CIDEr. On the full-video streaming task, StrAD-FT reaches a SODA score of 2.4 against 1.1 for our zero-shot baseline StrAD-Zero, though both exhibit limitations in temporal localization and narrative coherence. While prior work has tackled full-video AD generation in an offline, multi-stage fashion, ours is the first streaming approach, generating ADs on the fly without ground-truth timestamps. StrAD makes progress on full-video AD generation measurable, a prerequisite for scaling accessibility.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 38 of its 60 references that are in the map. First posted on arXiv 12 Aug 2026.
Builds on
38 of 60 references.
References · 38
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
- 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
- 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
- 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
- 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
- 2019Optuna: A Next-generation Hyperparameter Optimization Frameworkcited by 2,455
- 2014CIDEr: Consensus-based Image Description Evaluationcited by 2,027
- 2017Dense-Captioning Events in Videoscited by 789
- 2019CTRL: A Conditional Transformer Language Model for Controllable Generationcited by 754
- 2024VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMscited by 749
- 2023MovieChat: From Dense Token to Sparse Memory for Long Video Understandingcited by 536
- 2025Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAscited by 491
26 more
- 2023TimeChat: A Time-sensitive Multimodal Large Language Model for Long Video Understandingcited by 399
- 2023Vid2Seq: Large-Scale Pretraining of a Visual Language Model for Dense Video Captioningcited by 263
- 2015A Dataset for Movie Descriptioncited by 195
- 2016Movie Descriptioncited by 180
- 2023Powerset multi-class cross entropy loss for neural speaker diarizationcited by 109
- 2021MAD: A Scalable Dataset for Language Grounding in Videos from Movie Audio Descriptionscited by 107
- 2020Condensed Movies: Story Based Retrieval with Contextual Embeddingscited by 95
- 2024Streaming Dense Video Captioningcited by 82
- 2022Contrastive Search Is What You Need For Neural Text Generationcited by 58
- 2023AutoAD: Movie Description in Contextcited by 50
- 2023AutoAD II: The Sequel -- Who, When, and What in Movie Audio Descriptioncited by 48
- 2023MM-Narrator: Narrating Long-form Videos with Multimodal In-Context Learningcited by 46
- 2024MatchTime: Towards Automatic Soccer Game Commentary Generationcited by 37
- 2024AutoAD III: The Prequel -- Back to the Pixelscited by 35
- 2015Using Descriptive Video Services to Create a Large Data Source for Video Annotation Researchcited by 33
- 2024AutoAD-Zero: A Training-Free Framework for Zero-Shot Audio Descriptioncited by 21
- 2024Learning Video Context as Interleaved Multimodal Sequencescited by 16
- 2024Contextual AD Narration with Interleaved Multimodal Sequencecited by 14
- 2024LLM-AD: Large Language Model based Audio Description Systemcited by 14
- 2025Shot-by-Shot: Film-Grammar-Aware Training-Free Audio Description Generationcited by 13
- 2024DistinctAD: Distinctive Audio Description Generation in Contextscited by 11
- 2025What You See is What You Ask: Evaluating Audio Descriptionscited by 7
- 2025More than a Moment: Towards Coherent Sequences of Audio Descriptionscited by 5
- 2024NowYouSee Me: Context-Aware Automatic Audio Descriptioncited by 3
- 2025Character-Centric Understanding of Animated Moviescited by 3
- 2025MCAD: Multimodal Context-Aware Audio Description Generation For Soccercited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026REFRAMED: Towards Realistic Audio Description Generation for Movies
- 2026From Visual Cues to Spoken Narration: Rethinking Audio Description
- 2026StoryTeller: Training-Free Narrative Grounding for Long-Form Audio Description
- 2026TimeChat-Captioner: Scripting Multi-Scene Videos with Time-Aware and Structural Audio-Visual Captions
- 2026Script-a-Video: Deep Structured Audio-visual Captions via Factorized Streams and Relational Grounding
- 2026LongAV-Compass: Towards Unified Evaluation of Minute-Scale Audio-Visual Generation Across T2AV, I2AV, and V2AV
- 2024STA-V2A: Video-to-Audio Generation with Semantic and Temporal Alignment
- 2024Movie101v2: Improved Movie Narration Benchmark