mlsanityData to 14 Oct 2026

Condensed Movies: Story Based Retrieval with Contextual Embeddings

arXiv 2005.04208 ↗First posted 8 May 2020cs.CVcited by 95

Not on the map. Why?

Abstract (arXiv)

Our objective in this work is long range understanding of the narrative structure of movies. Instead of considering the entire movie, we propose to learn from the `key scenes' of the movie, providing a condensed look at the full storyline. To this end, we make the following three contributions: (i) We create the Condensed Movies Dataset (CMD) consisting of the key scenes from over 3K movies: each key scene is accompanied by a high level semantic description of the scene, character face-tracks, and metadata about the movie. The dataset is scalable, obtained automatically from YouTube, and is freely available for anybody to download and use. It is also an order of magnitude larger than existing movie datasets in the number of movies; (ii) We provide a deep network baseline for text-to-video retrieval on our dataset, combining character, speech and visual cues into a single video embedding; and finally (iii) We demonstrate how the addition of context from other video clips improves retrieval performance.

Key facts. Cited by 95 papers in the map (data through 14 Oct 2026). First posted on arXiv 8 May 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 10; 2022: 15; 2023: 14; 2024: 24; 2025: 13; 2026: 19.

95 papers.

Sort:

By area:

  1. 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,177
  2. 2021Towards Long-Form Video Understandingcited by 146
  3. 2024MiniGPT4-Video: Advancing Multimodal LLMs for Video Understanding with Interleaved Visual-Textual Tokenscited by 118
  4. 2024CinePile: A Long Video Question Answering Dataset and Benchmarkcited by 114
  5. 2022Long Movie Clip Classification with State-Space Video Modelscited by 106
  6. 2024MovieDreamer: Hierarchical Generation for Coherent Long Visual Sequencecited by 89
  7. 2022Learning Audio-Video Modalities from Image Captionscited by 80
  8. 2021Visual Semantic Role Labeling for Video Understandingcited by 60
  9. 2022Long-Form Video-Language Pre-Training with Multimodal Temporal Contrastive Learningcited by 59
  10. 2022A CLIP-Hitchhiker's Guide to Long Video Retrievalcited by 57
  11. 2023AutoAD: Movie Description in Contextcited by 50
  12. 2023AutoAD II: The Sequel -- Who, When, and What in Movie Audio Descriptioncited by 49
38 more
  1. 2023TESTA: Temporal-Spatial Token Aggregation for Long-form Video-Language Understandingcited by 49
  2. 2021On Semantic Similarity in Video Retrievalcited by 46
  3. 2024Goldfish: Vision-Language Understanding of Arbitrarily Long Videoscited by 40
  4. 2024AutoAD III: The Prequel -- Back to the Pixelscited by 35
  5. 2025Temporal Working Memory: Query-Guided Segment Refinement for Enhanced Multimodal Understandingcited by 28
  6. 2021MovieCuts: A New Dataset and Benchmark for Cut Type Recognitioncited by 26
  7. 2022The Anatomy of Video Editing: A Dataset and Benchmark Suite for AI-Assisted Video Editingcited by 25
  8. 2024E.T. the Exceptional Trajectories: Text-to-camera-trajectory generation with character awarenesscited by 25
  9. 2021Face, Body, Voice: Video Person-Clustering with Multiple Modalitiescited by 24
  10. 2021Masking Modalities for Cross-modal Video Retrievalcited by 23
  11. 2024AutoAD-Zero: A Training-Free Framework for Zero-Shot Audio Descriptioncited by 23
  12. 2022Hierarchical Self-supervised Representation Learning for Movie Understandingcited by 19
  13. 2021Learning to Cut by Watching Moviescited by 18
  14. 2024What Makes a Good Story and How Can We Measure It? A Comprehensive Survey of Story Evaluationcited by 17
  15. 2022MovieCLIP: Visual Scene Recognition in Moviescited by 16
  16. 2024Learning Video Context as Interleaved Multimodal Sequencescited by 16
  17. 2025Audio-Sync Video Generation with Multi-Stream Temporal Controlcited by 15
  18. 2025Shot-by-Shot: Film-Grammar-Aware Training-Free Audio Description Generationcited by 13
  19. 2021Finding the Right Moment: Human-Assisted Trailer Creation via Task Compositioncited by 12
  20. 2022Match Cutting: Finding Cuts with Smooth Visual Transitionscited by 12
  21. 2024Long Story Short: Story-level Video Understanding from 20K Short Filmscited by 12
  22. 2024DistinctAD: Distinctive Audio Description Generation in Contextscited by 12
  23. 2022Weakly-Supervised Temporal Article Groundingcited by 11
  24. 2024Tails Tell Tales: Chapter-Wide Manga Transcriptions with Character Namescited by 11
  25. 2022Multi-modal Segment Assemblage Network for Ad Video Editing with Importance-Coherence Rewardcited by 10
  26. 2021Automated Video Labelling: Identifying Faces by Corroborative Evidencecited by 9
  27. 2024Movie101v2: Improved Movie Narration Benchmarkcited by 9
  28. 2024A Survey of Video Datasets for Grounded Event Understandingcited by 9
  29. 2025STORYANCHORS: Generating Consistent Multi-Scene Story Frames for Long-Form Narrativescited by 9
  30. 2025Video Color Grading via Look-Up Table Generationcited by 9
  31. 2025AVI-Edit: Audio-sync Video Instance Editing with Granularity-Aware Mask Refinercited by 9
  32. 2022Synopses of Movie Narratives: a Video-Language Dataset for Story Understandingcited by 8
  33. 2023MM-AU:Towards Multimodal Understanding of Advertisement Videoscited by 8
  34. 2025What You See is What You Ask: Evaluating Audio Descriptionscited by 8
  35. 2025Pulp Motion: Framing-aware multimodal camera and human motion generationcited by 8
  36. 2023Long-range Multimodal Pretraining for Movie Understandingcited by 7
  37. 2023Revisiting Kernel Temporal Segmentation as an Adaptive Tokenizer for Long-form Video Understandingcited by 7
  38. 2023How you feelin'? Learning Emotions and Mental States in Movie Scenescited by 6

50 most cited of 95 All 95 (JSON)