mlsanityData to 7 Oct 2026

Movie Description

arXiv 1605.03705 ↗First posted 12 May 2016cs.CV cs.CLcited by 180

Not on the map. Why?

Abstract (arXiv)

Audio Description (AD) provides linguistic descriptions of movies and allows visually impaired people to follow a movie along with their peers. Such descriptions are by design mainly visual and thus naturally form an interesting data source for computer vision and computational linguistics. In this work we propose a novel dataset which contains transcribed ADs, which are temporally aligned to full length movies. In addition we also collected and aligned movie scripts used in prior work and compare the two sources of descriptions. In total the Large Scale Movie Description Challenge (LSMDC) contains a parallel corpus of 118,114 sentences and video clips from 202 movies. First we characterize the dataset by benchmarking different approaches for generating video descriptions. Comparing ADs to scripts, we find that ADs are indeed more visual and describe precisely what is shown rather than what should happen according to the scripts created prior to movie production. Furthermore, we present and compare the results of several teams who participated in a challenge organized in the context of the workshop "Describing and Understanding Video & The Large Scale Movie Description Challenge (LSMDC)", at ICCV 2015.

Key facts. Cited by 180 papers in the map (data through 7 Oct 2026). First posted on arXiv 12 May 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 25; 2022: 34; 2023: 31; 2024: 39; 2025: 28; 2026: 23.

180 papers.

Sort:

By area:

  1. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,388
  2. 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,175
  3. 2023InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and Generationcited by 488
  4. 2024VideoMamba: State Space Model for Efficient Video Understandingcited by 325
  5. 2021MERLOT: Multimodal Neural Script Knowledge Modelscited by 311
  6. 2021Prompting Visual-Language Models for Efficient Video Understandingcited by 309
  7. 2022Zero-Shot Video Question Answering via Frozen Bidirectional Language Modelscited by 229
  8. 2021Advancing High-Resolution Video-Language Representation with Large-Scale Video Transcriptionscited by 226
  9. 2022Learning Video Representations from Large Language Modelscited by 204
  10. 2023A Survey on Video Diffusion Modelscited by 203
  11. 2023Unmasked Teacher: Towards Training-Efficient Video Foundation Modelscited by 199
  12. 2022MERLOT Reserve: Neural Script Knowledge through Vision and Language and Soundcited by 184
38 more
  1. 2023VAST: A Vision-Audio-Subtitle-Text Omni-Modality Foundation Model and Datasetcited by 168
  2. 2023Compositional Exemplars for In-context Learningcited by 143
  3. 2024Tarsier: Recipes for Training and Evaluating Large Video Description Modelscited by 140
  4. 2023VALOR: Vision-Audio-Language Omni-Perception Pretraining Model and Datasetcited by 138
  5. 2022X-Pool: Cross-Modal Language-Video Attention for Text-Video Retrievalcited by 127
  6. 2021Improving Video-Text Retrieval by Multi-Stream Corpus Alignment and Dual Softmax Losscited by 116
  7. 2021MAD: A Scalable Dataset for Language Grounding in Videos from Movie Audio Descriptionscited by 107
  8. 2022TS2-Net: Token Shift and Selection Transformer for Text-Video Retrievalcited by 104
  9. 2023Vita-CLIP: Video and text adaptive CLIP via Multimodal Promptingcited by 90
  10. 2024Video ReCap: Recursive Captioning of Hour-Long Videoscited by 88
  11. 2021SUTD-TrafficQA: A Question Answering Benchmark and an Efficient Network for Video Reasoning over Traffic Eventscited by 84
  12. 2024Mono-InternVL: Pushing the Boundaries of Monolithic Multimodal Large Language Models with Endogenous Visual Pre-trainingcited by 82
  13. 2021TEACHTEXT: CrossModal Generalized Distillation for Text-Video Retrievalcited by 81
  14. 2022Learning Audio-Video Modalities from Image Captionscited by 80
  15. 2021MDMMT: Multidomain Multimodal Transformer for Video Retrievalcited by 78
  16. 2023Verbs in Action: Improving verb understanding in video-language modelscited by 76
  17. 2022CLIP-ViP: Adapting Pre-trained Image-Text Model to Video-Language Representation Alignmentcited by 74
  18. 2024A Survey of Multimodal Large Language Model from A Data-centric Perspectivecited by 73
  19. 2021Cross Modal Retrieval with Querybank Normalisationcited by 71
  20. 2021Object-aware Video-language Pre-training for Retrievalcited by 57
  21. 2023Revisiting Temporal Modeling for CLIP-based Image-to-Video Knowledge Transferringcited by 57
  22. 2021Spoken Moments: Learning Joint Audio-Visual Representations from Video Descriptionscited by 53
  23. 2023AutoAD: Movie Description in Contextcited by 50
  24. 2023NaQ: Leveraging Narrations as Queries to Supervise Episodic Memorycited by 46
  25. 2023MM-Narrator: Narrating Long-form Videos with Multimodal In-Context Learningcited by 46
  26. 2023Paxion: Patching Action Knowledge in Video-Language Foundation Modelscited by 42
  27. 2023BT-Adapter: Video Conversation is Feasible Without Video Instruction Tuningcited by 41
  28. 2024vid-TLDR: Training Free Token merging for Light-weight Video Transformercited by 41
  29. 2023Benchmarks for Automated Commonsense Reasoning: A Surveycited by 40
  30. 2022Look for the Change: Learning Object States and State-Modifying Actions from Untrimmed Web Videoscited by 39
  31. 2024AutoAD III: The Prequel -- Back to the Pixelscited by 35
  32. 2025Visual Large Language Models for Generalized and Specialized Applicationscited by 35
  33. 2025Mavors: Multi-granularity Video Representation for Multimodal Large Language Modelcited by 31
  34. 2023A Review of Deep Learning for Video Captioningcited by 28
  35. 2022CONE: An Efficient COarse-to-fiNE Alignment Framework for Long Video Temporal Groundingcited by 27
  36. 2021MovieCuts: A New Dataset and Benchmark for Cut Type Recognitioncited by 26
  37. 2024From Sora What We Can See: A Survey of Text-to-Video Generationcited by 26
  38. 2023Movie101: A New Movie Understanding Benchmarkcited by 25

50 most cited of 180 All 180 (JSON)