mlsanityData to 14 Oct 2026

FocusedAD: Character-centric Movie Audio Description

arXiv 2504.12157 ↗First posted 16 Apr 2025cs.CVcited by 3

Not on the map. Why?

Abstract (arXiv)

Movie Audio Description (AD) aims to narrate visual content during dialogue-free segments, particularly benefiting blind and visually impaired (BVI) audiences. Compared with general video captioning, AD demands plot-relevant narration with explicit character name references, posing unique challenges in movie understanding.To identify active main characters and focus on storyline-relevant regions, we propose FocusedAD, a novel framework that delivers character-centric movie audio descriptions. It includes: (i) a Character Perception Module(CPM) for tracking character regions and linking them to names; (ii) a Dynamic Prior Module(DPM) that injects contextual cues from prior ADs and subtitles via learnable soft prompts; and (iii) a Focused Caption Module(FCM) that generates narrations enriched with plot-relevant details and named characters. To overcome limitations in character identification, we also introduce an automated pipeline for building character query banks. FocusedAD achieves state-of-the-art performance on multiple benchmarks, including strong zero-shot results on MAD-eval-Named and our newly proposed Cinepile-AD dataset. Code and data will be released at https://github.com/Thorin215/FocusedAD .

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 33 of its 55 references that are in the map. First posted on arXiv 16 Apr 2025.

Builds on

33 of 55 references.

References · 33

  1. 2025Qwen2.5-VL Technical Reportcited by 5,587
  2. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  3. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  4. 2016Feature Pyramid Networks for Object Detectioncited by 5,036
  5. 2015FaceNet: A Unified Embedding for Face Recognition and Clusteringcited by 2,849
  6. 2016Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networkscited by 903
  7. 2016SPICE: Semantic Propositional Image Caption Evaluationcited by 824
  8. 2017Dense-Captioning Events in Videoscited by 793
  9. 2024ShareGPT4Video: Improving Video Understanding and Generation with Better Captionscited by 388
  10. 2023Hiera: A Hierarchical Vision Transformer without the Bells-and-Whistlescited by 290
  11. 2023Vid2Seq: Large-Scale Pretraining of a Visual Language Model for Dense Video Captioningcited by 263
  12. 2020UniVL: A Unified Video and Language Pre-Training Model for Multimodal Understanding and Generationcited by 259
21 more
  1. 2020MovieNet: A Holistic Dataset for Movie Understandingcited by 206
  2. 2016Joint Detection and Identification Feature Learning for Person Searchcited by 190
  3. 2022CLIP-ReID: Exploiting Vision-Language Model for Image Re-Identification without Concrete Text Labelscited by 181
  4. 2021SwinBERT: End-to-End Transformers with Sparse Attention for Video Captioningcited by 162
  5. 2022End-to-end Generative Pretraining for Multimodal Video Captioningcited by 116
  6. 2024CinePile: A Long Video Question Answering Dataset and Benchmarkcited by 114
  7. 2023MM-VID: Advancing Video Understanding with GPT-4V(ision)cited by 83
  8. 2024Streaming Dense Video Captioningcited by 82
  9. 2024VideoRefer Suite: Advancing Spatial-Temporal Object Understanding with Video LLMcited by 72
  10. 2018Jointly Localizing and Describing Events for Dense Video Captioningcited by 60
  11. 2019Streamlined Dense Video Captioningcited by 57
  12. 2023AutoAD II: The Sequel -- Who, When, and What in Movie Audio Descriptioncited by 49
  13. 2023MM-Narrator: Narrating Long-form Videos with Multimodal In-Context Learningcited by 46
  14. 2023Movie101: A New Movie Understanding Benchmarkcited by 25
  15. 2021Face, Body, Voice: Video Person-Clustering with Multiple Modalitiescited by 24
  16. 2020Learning Interactions and Relationships between Movie Characterscited by 22
  17. 2018From Benedict Cumberbatch to Sherlock Holmes: Character Identification in TV series without a Scriptcited by 20
  18. 2020iPerceive: Applying Common-Sense Reasoning to Multi-Modal Dense Video Captioning and Video Question Answeringcited by 20
  19. 2018Person Search in Videos with One Portrait Through Visual and Temporal Linkscited by 14
  20. 2021Automated Video Labelling: Identifying Faces by Corroborative Evidencecited by 9
  21. 2024Learning Long-form Video Prior via Generative Pre-Trainingcited by 8

Built on by

Cited by year

Citing papers per year: 2025: 3.

3 papers.