mlsanityData to 14 Oct 2026

MovieNet: A Holistic Dataset for Movie Understanding

arXiv 2007.10937 ↗First posted 21 Jul 2020cs.CVcited by 206

Not on the map. Why?

Cited by 42 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Recent years have seen remarkable advances in visual understanding. However, how to understand a story-based long video with artistic styles, e.g. movie, remains challenging. In this paper, we introduce MovieNet -- a holistic dataset for movie understanding. MovieNet contains 1,100 movies with a large amount of multi-modal data, e.g. trailers, photos, plot descriptions, etc. Besides, different aspects of manual annotations are provided in MovieNet, including 1.1M characters with bounding boxes and identities, 42K scene boundaries, 2.5K aligned description sentences, 65K tags of place and action, and 92K tags of cinematic style. To the best of our knowledge, MovieNet is the largest dataset with richest annotations for comprehensive movie understanding. Based on MovieNet, we set up several benchmarks for movie understanding from different angles. Extensive experiments are executed on these benchmarks to show the immeasurable value of MovieNet and the gap of current approaches towards comprehensive movie understanding. We believe that such a holistic dataset would promote the researches on story-based long video understanding and beyond. MovieNet will be published in compliance with regulations at https://movienet.github.io.

Key facts. Cited by 206 papers in the map (data through 14 Oct 2026). First posted on arXiv 21 Jul 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 18; 2022: 27; 2023: 23; 2024: 58; 2025: 44; 2026: 36.

206 papers.

Sort:

By area:

  1. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,652
  2. 2023MVBench: A Comprehensive Multi-modal Video Understanding Benchmarkcited by 1,109
  3. 2023EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understandingcited by 680
  4. 2023LLaMA-VID: An Image is Worth 2 Tokens in Large Language Modelscited by 546
  5. 2023MovieChat: From Dense Token to Sparse Memory for Long Video Understandingcited by 542
  6. 2024MLVU: Benchmarking Multi-task Long Video Understandingcited by 292
  7. 2023Video Understanding with Large Language Models: A Surveycited by 206
  8. 2024Flash-VStream: Memory-Based Real-Time Understanding for Long Video Streamscited by 169
  9. 2024Streaming Long Video Understanding with Large Language Modelscited by 167
  10. 2024Oryx MLLM: On-Demand Spatial-Temporal Understanding at Arbitrary Resolutioncited by 159
  11. 2021Towards Long-Form Video Understandingcited by 146
  12. 2024StreamingBench: Assessing the Gap for MLLMs to Achieve Streaming Video Understandingcited by 144
38 more
  1. 2025OVO-Bench: How Far is Your Video-LLMs from Real-World Online Video Understanding?cited by 142
  2. 2024RLAIF-V: Open-Source AI Feedback Leads to Super GPT-4V Trustworthinesscited by 131
  3. 2025Streaming Video Question-Answering with In-context Video KV-Cache Retrievalcited by 117
  4. 2024HourVideo: 1-Hour Video-Language Understandingcited by 113
  5. 2022Long Movie Clip Classification with State-Space Video Modelscited by 106
  6. 2024Mini-InternVL: A Flexible-Transfer Pocket Multimodal Model with 5% Parameters and 90% Performancecited by 94
  7. 2024MileBench: Benchmarking MLLMs in Long Contextcited by 88
  8. 2025Flash-VStream: Efficient Real-Time Understanding for Long Video Streamscited by 81
  9. 2024MovieChat+: Question-aware Sparse Memory for Long Video Question Answeringcited by 73
  10. 2021Category-Level 6D Object Pose Estimation via Cascaded Relation and Recurrent Reconstruction Networkscited by 69
  11. 2024DrVideo: Document Retrieval Based Long Video Understandingcited by 65
  12. 2025Eagle 2.5: Boosting Long-Context Post-Training for Frontier Vision-Language Modelscited by 64
  13. 2025Towards Understanding Camera Motions in Any Videocited by 64
  14. 2021Visual Semantic Role Labeling for Video Understandingcited by 60
  15. 2024Needle In A Multimodal Haystackcited by 58
  16. 2025InfiniPot-V: Memory-Constrained KV Cache Compression for Streaming Video Understandingcited by 58
  17. 2025SVBench: A Benchmark with Temporal Multi-Turn Dialogues for Streaming Video Understandingcited by 57
  18. 2022Efficient Movie Scene Detection using State-Space Transformerscited by 51
  19. 2021Learning the Predictability of the Futurecited by 49
  20. 2023AutoAD II: The Sequel -- Who, When, and What in Movie Audio Descriptioncited by 49
  21. 2021Large-Scale Spatio-Temporal Person Re-identification: Algorithms and Benchmarkcited by 47
  22. 2026HERMES: KV Cache as Hierarchical Memory for Efficient Streaming Video Understandingcited by 41
  23. 2024Goldfish: Vision-Language Understanding of Arbitrarily Long Videoscited by 40
  24. 2024Align Anything: Training All-Modality Models to Follow Instructions with Language Feedbackcited by 39
  25. 2025Long-VITA: Scaling Large Multi-modal Models to 1 Million Tokens with Leading Short-Context Accuracycited by 39
  26. 2021Graph Sampling Based Deep Metric Learning for Generalizable Person Re-Identificationcited by 37
  27. 2024MovieBench: A Hierarchical Movie Level Dataset for Long Video Generationcited by 36
  28. 2025VideoGen-of-Thought: Step-by-step generating multi-shot video with minimal manual interventioncited by 36
  29. 2021Shot Contrastive Self-Supervised Learning for Scene Boundary Detectioncited by 35
  30. 2024AutoAD III: The Prequel -- Back to the Pixelscited by 35
  31. 2025StreamAgent: Towards Anticipatory Agents for Streaming Video Understandingcited by 33
  32. 2023Visual Writing Prompts: Character-Grounded Story Generation with Curated Image Sequencescited by 31
  33. 2022The One Where They Reconstructed 3D Humans and Environments in TV Showscited by 30
  34. 2024A Comprehensive Survey of Direct Preference Optimization: Datasets, Theories, Variants, and Applicationscited by 27
  35. 2021MovieCuts: A New Dataset and Benchmark for Cut Type Recognitioncited by 26
  36. 2022The Anatomy of Video Editing: A Dataset and Benchmark Suite for AI-Assisted Video Editingcited by 25
  37. 2023LvBench: A Benchmark for Long-form Video Understanding with Versatile Multi-modal Question Answeringcited by 25
  38. 2024PVC: Progressive Visual Token Compression for Unified Image and Video Processing in Large Vision-Language Modelscited by 25

50 most cited of 206 All 206 (JSON)