mlsanityData to 7 Oct 2026

StrAD: A Streaming Method and Benchmark for Audio Description Generation for Long-form Videos

arXiv 2608.12549 ↗First posted 12 Aug 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Visual content is the dominant medium of communication, yet without audio descriptions (ADs), it remains inaccessible to blind and low-vision people. ADs narrate context-relevant visual events during natural audio pauses. Manually creating ADs is expensive, limiting coverage to a small fraction of available content. Most existing automatic AD generation methods frame the task as video clip captioning, requiring ground-truth timestamps and additional context cues such as character databases. Current benchmarks reinforce this framing, consisting of short video segments paired with automatic or task-mismatched annotations. We introduce StrAD, a benchmark for long-form AD generation on full-length videos spanning diverse genres such as movies, documentaries, short films, performances, and video games. We reformulate AD generation as streaming dense video captioning. Our approach processes full-length videos with a sliding window, inserting ADs into existing transcripts without ground-truth timestamps, and supports both fine-tuned models and zero-shot prompting of vision-language models. On the segment-level task with given timestamps, our fine-tuned StrAD-FT sets the state of the art on CMD-AD with 36.3 CIDEr (+10.0 over Shot-by-shot), establishes a reference point on StrAD (51.0 CIDEr), and remains competitive on MAD-Eval at 24.9 CIDEr. On the full-video streaming task, StrAD-FT reaches a SODA score of 2.4 against 1.1 for our zero-shot baseline StrAD-Zero, though both exhibit limitations in temporal localization and narrative coherence. While prior work has tackled full-video AD generation in an offline, multi-stage fashion, ours is the first streaming approach, generating ADs on the fly without ground-truth timestamps. StrAD makes progress on full-video AD generation measurable, a prerequisite for scaling accessibility.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 38 of its 60 references that are in the map. First posted on arXiv 12 Aug 2026.

Builds on

38 of 60 references.

References · 38

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  3. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  4. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  5. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  6. 2019Optuna: A Next-generation Hyperparameter Optimization Frameworkcited by 2,455
  7. 2014CIDEr: Consensus-based Image Description Evaluationcited by 2,027
  8. 2017Dense-Captioning Events in Videoscited by 789
  9. 2019CTRL: A Conditional Transformer Language Model for Controllable Generationcited by 754
  10. 2024VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMscited by 749
  11. 2023MovieChat: From Dense Token to Sparse Memory for Long Video Understandingcited by 536
  12. 2025Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAscited by 491
26 more
  1. 2023TimeChat: A Time-sensitive Multimodal Large Language Model for Long Video Understandingcited by 399
  2. 2023Vid2Seq: Large-Scale Pretraining of a Visual Language Model for Dense Video Captioningcited by 263
  3. 2015A Dataset for Movie Descriptioncited by 195
  4. 2016Movie Descriptioncited by 180
  5. 2023Powerset multi-class cross entropy loss for neural speaker diarizationcited by 109
  6. 2021MAD: A Scalable Dataset for Language Grounding in Videos from Movie Audio Descriptionscited by 107
  7. 2020Condensed Movies: Story Based Retrieval with Contextual Embeddingscited by 95
  8. 2024Streaming Dense Video Captioningcited by 82
  9. 2022Contrastive Search Is What You Need For Neural Text Generationcited by 58
  10. 2023AutoAD: Movie Description in Contextcited by 50
  11. 2023AutoAD II: The Sequel -- Who, When, and What in Movie Audio Descriptioncited by 48
  12. 2023MM-Narrator: Narrating Long-form Videos with Multimodal In-Context Learningcited by 46
  13. 2024MatchTime: Towards Automatic Soccer Game Commentary Generationcited by 37
  14. 2024AutoAD III: The Prequel -- Back to the Pixelscited by 35
  15. 2015Using Descriptive Video Services to Create a Large Data Source for Video Annotation Researchcited by 33
  16. 2024AutoAD-Zero: A Training-Free Framework for Zero-Shot Audio Descriptioncited by 21
  17. 2024Learning Video Context as Interleaved Multimodal Sequencescited by 16
  18. 2024Contextual AD Narration with Interleaved Multimodal Sequencecited by 14
  19. 2024LLM-AD: Large Language Model based Audio Description Systemcited by 14
  20. 2025Shot-by-Shot: Film-Grammar-Aware Training-Free Audio Description Generationcited by 13
  21. 2024DistinctAD: Distinctive Audio Description Generation in Contextscited by 11
  22. 2025What You See is What You Ask: Evaluating Audio Descriptionscited by 7
  23. 2025More than a Moment: Towards Coherent Sequences of Audio Descriptionscited by 5
  24. 2024NowYouSee Me: Context-Aware Automatic Audio Descriptioncited by 3
  25. 2025Character-Centric Understanding of Animated Moviescited by 3
  26. 2025MCAD: Multimodal Context-Aware Audio Description Generation For Soccercited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026REFRAMED: Towards Realistic Audio Description Generation for Movies
  2. 2026From Visual Cues to Spoken Narration: Rethinking Audio Description
  3. 2026StoryTeller: Training-Free Narrative Grounding for Long-Form Audio Description
  4. 2026TimeChat-Captioner: Scripting Multi-Scene Videos with Time-Aware and Structural Audio-Visual Captions
  5. 2026Script-a-Video: Deep Structured Audio-visual Captions via Factorized Streams and Relational Grounding
  6. 2026LongAV-Compass: Towards Unified Evaluation of Minute-Scale Audio-Visual Generation Across T2AV, I2AV, and V2AV
  7. 2024STA-V2A: Video-to-Audio Generation with Semantic and Temporal Alignment
  8. 2024Movie101v2: Improved Movie Narration Benchmark