mlsanityData to 7 Oct 2026

StoryTeller: Training-Free Narrative Grounding for Long-Form Audio Description

arXiv 2607.11798 ↗First posted 13 Jul 2026cs.CV cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Long-form audio description (AD) requires more than describing visible actions: it must preserve characters, events, relationships, and story context across scenes so that blind and low-vision (BLV) audiences can follow a film. Modern video-language models (VLMs) are effective on short clips, but they often treat each moment independently, producing descriptions that miss who characters are, why events matter, and how the current scene connects to earlier narrative context. We propose StoryTeller, a training-free framework for story-aware long-form AD. Instead of relying only on local visual cues, StoryTeller maintains a verified narrative memory that carries forward story-relevant information across scenes, enabling later descriptions to remain coherent, grounded, and contextually informative. Given only raw video and a movie title, StoryTeller can optionally retrieve public movie metadata to resolve names and story context, while accepting only facts that are supported by the video through semantic filtering and VLM verification. The method requires no subtitles, scripts, AD transcripts, aligned captions, character banks, precomputed face identities, or task-specific fine-tuning. To evaluate whether generated AD preserves narrative information, we introduce StoryAD-QA, a question-answering benchmark that tests whether a language model can answer story-context questions using only the generated descriptions. Experiments on standard AD benchmarks and diverse long-form videos show that StoryTeller consistently improves narrative coherence, factual grounding, and story comprehension over strong baselines in automatic, QA-based, and human evaluations.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 33 of its 39 references that are in the map. First posted on arXiv 13 Jul 2026.

Builds on

33 of 39 references.

References · 33

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  2. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  3. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
  4. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  5. 2014CIDEr: Consensus-based Image Description Evaluationcited by 2,027
  6. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
  7. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,203
  8. 2016SPICE: Semantic Propositional Image Caption Evaluationcited by 820
  9. 2024VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMscited by 749
  10. 2023MovieChat: From Dense Token to Sparse Memory for Long Video Understandingcited by 536
  11. 2018TVQA: Localized, Compositional Video Question Answeringcited by 463
  12. 2015MovieQA: Understanding Stories in Movies through Question-Answeringcited by 304
21 more
  1. 2024Visual CoT: Advancing Multi-Modal Language Models with a Comprehensive Dataset and Benchmark for Chain-of-Thought Reasoningcited by 267
  2. 2024VideoAgent: A Memory-augmented Multimodal Agent for Video Understandingcited by 224
  3. 2026Qwen3-VL-Embedding and Qwen3-VL-Reranker: A Unified Framework for State-of-the-Art Multimodal Retrieval and Rankingcited by 211
  4. 2020MovieNet: A Holistic Dataset for Movie Understandingcited by 206
  5. 2016Movie Descriptioncited by 180
  6. 2024Video-RAG: Visually-aligned Retrieval-Augmented Long Video Comprehensioncited by 134
  7. 2021MAD: A Scalable Dataset for Language Grounding in Videos from Movie Audio Descriptionscited by 107
  8. 2025Seeing, Listening, Remembering, and Reasoning: A Multimodal Agent with Long-Term Memorycited by 88
  9. 2024Optimus-1: Hybrid Multimodal Memory Empowered Agents Excel in Long-Horizon Taskscited by 83
  10. 2023MM-VID: Advancing Video Understanding with GPT-4V(ision)cited by 82
  11. 2017MovieGraphs: Towards Understanding Human-Centric Situations from Videoscited by 67
  12. 2023AutoAD: Movie Description in Contextcited by 50
  13. 2023AutoAD II: The Sequel -- Who, When, and What in Movie Audio Descriptioncited by 48
  14. 2023MM-Narrator: Narrating Long-form Videos with Multimodal In-Context Learningcited by 46
  15. 2024AutoAD III: The Prequel -- Back to the Pixelscited by 35
  16. 2023Movie101: A New Movie Understanding Benchmarkcited by 25
  17. 2024AutoAD-Zero: A Training-Free Framework for Zero-Shot Audio Descriptioncited by 21
  18. 2024StoryTeller: Improving Long Video Description through Global Audio-Visual Character Identificationcited by 13
  19. 2025Video-EM: Event-Centric Episodic Memory for Long-Form Video Understandingcited by 10
  20. 2024MICap: A Unified Model for Identity-aware Movie Descriptionscited by 9
  21. 2025What You See is What You Ask: Evaluating Audio Descriptionscited by 7

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026REFRAMED: Towards Realistic Audio Description Generation for Movies
  2. 2026From Visual Cues to Spoken Narration: Rethinking Audio Description
  3. 2026StrAD: A Streaming Method and Benchmark for Audio Description Generation for Long-form Videos
  4. 2024Movie101v2: Improved Movie Narration Benchmark
  5. 2024DistinctAD: Distinctive Audio Description Generation in Contexts
  6. 2026MovieTeller: Tool-augmented Movie Synopsis with ID Consistent Progressive Abstraction
  7. 2026Script-a-Video: Deep Structured Audio-visual Captions via Factorized Streams and Relational Grounding
  8. 2025Movie Facts and Fibs (MF2): A Benchmark for Long Movie Understanding