mlsanityData to 14 Oct 2026

MovieQA: Understanding Stories in Movies through Question-Answering

arXiv 1512.02902 ↗First posted 9 Dec 2015cs.CV cs.CLcited by 304

Not on the map. Why?

Cited by 40 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We introduce the MovieQA dataset which aims to evaluate automatic story comprehension from both video and text. The dataset consists of 14,944 questions about 408 movies with high semantic diversity. The questions range from simpler "Who" did "What" to "Whom", to "Why" and "How" certain events occurred. Each question comes with a set of five possible answers; a correct one and four deceiving answers provided by human annotators. Our dataset is unique in that it contains multiple sources of information -- video clips, plots, subtitles, scripts, and DVS. We analyze our data through various statistics and methods. We further extend existing QA techniques to show that question-answering with such open-ended semantics is hard. We make this data set public along with an evaluation benchmark to encourage inspiring work in this challenging domain.

Key facts. Cited by 304 papers in the map (data through 14 Oct 2026). First posted on arXiv 9 Dec 2015.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2018: 1; 2019: 0; 2020: 0; 2021: 58; 2022: 61; 2023: 51; 2024: 59; 2025: 43; 2026: 31.

304 papers.

Sort:

By area:

  1. 2021NExT-QA:Next Phase of Question-Answering to Explaining Temporal Actionscited by 744
  2. 2022A-OKVQA: A Benchmark for Visual Question Answering using World Knowledgecited by 710
  3. 2023EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understandingcited by 680
  4. 2023MovieChat: From Dense Token to Sparse Memory for Long Video Understandingcited by 542
  5. 2024LVBench: An Extreme Long Video Understanding Benchmarkcited by 410
  6. 2023The Belebele Benchmark: a Parallel Reading Comprehension Dataset in 122 Language Variantscited by 311
  7. 2018Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigationcited by 305
  8. 2022Zero-Shot Video Question Answering via Frozen Bidirectional Language Modelscited by 229
  9. 2024VideoAgent: A Memory-augmented Multimodal Agent for Video Understandingcited by 229
  10. 2024STAR: A Benchmark for Situated Reasoning in Real-World Videoscited by 213
  11. 2022Learning to Answer Questions in Dynamic Audio-Visual Scenarioscited by 191
  12. 2024MMBench-Video: A Long-Form Multi-Shot Benchmark for Holistic Video Understandingcited by 182
38 more
  1. 2023SlideVQA: A Dataset for Document Visual Question Answering on Multiple Imagescited by 179
  2. 2023A Simple LLM Framework for Long-Range Video Question-Answeringcited by 178
  3. 2021Towards Long-Form Video Understandingcited by 146
  4. 2024Momentor: Advancing Video Large Language Model with Fine-Grained Temporal Reasoningcited by 122
  5. 2021AGQA: A Benchmark for Compositional Spatio-Temporal Reasoningcited by 120
  6. 2022Foundations and Trends in Multimodal Machine Learning: Principles, Challenges, and Open Questionscited by 116
  7. 2022EgoTaskQA: Understanding Human Tasks in Egocentric Videoscited by 115
  8. 2024CinePile: A Long Video Question Answering Dataset and Benchmarkcited by 114
  9. 2021WebQA: Multihop and Multimodal QAcited by 109
  10. 2023Video-Bench: A Comprehensive Benchmark and Toolkit for Evaluating Video-based Large Language Modelscited by 107
  11. 2022Long Movie Clip Classification with State-Space Video Modelscited by 106
  12. 2021TACo: Token-aware Cascade Contrastive Learning for Video-Text Alignmentcited by 102
  13. 2021WebSRC: A Dataset for Web-Based Structural Reading Comprehensioncited by 100
  14. 2021QA Dataset Explosion: A Taxonomy of NLP Resources for Question Answering and Reading Comprehensioncited by 91
  15. 2021Pano-AVQA: Grounded Audio-Visual Question Answering on 360^circ Videoscited by 86
  16. 2021SUTD-TrafficQA: A Question Answering Benchmark and an Efficient Network for Video Reasoning over Traffic Eventscited by 84
  17. 2022Video Question Answering: Datasets, Algorithms and Challengescited by 84
  18. 2024Natural Language Understanding and Inference with MLLM in Visual Question Answering: A Surveycited by 76
  19. 2024MovieChat+: Question-aware Sparse Memory for Long Video Question Answeringcited by 73
  20. 2023Vista-LLaMA: Reducing Hallucination in Video Language Models via Equal Distance to Visual Tokenscited by 71
  21. 2021Grounding Physical Concepts of Objects and Events Through Dynamic Visual Reasoningcited by 68
  22. 2024Too Many Frames, Not All Useful: Efficient Strategies for Long-Form Video QAcited by 68
  23. 2023HierVL: Learning Hierarchical Video-Language Embeddingscited by 66
  24. 2022ComPhy: Compositional Physical Reasoning of Objects and Events from Videoscited by 63
  25. 2021Visual Semantic Role Labeling for Video Understandingcited by 60
  26. 2024DoraemonGPT: Toward Understanding Dynamic Scenes with Large Language Models (Exemplified as A Video Agent)cited by 58
  27. 2024Needle In A Multimodal Haystackcited by 58
  28. 2022Cross-Modal Causal Relational Reasoning for Event-Level Visual Question Answeringcited by 57
  29. 2023SoccerNet-Caption: Dense Video Captioning for Soccer Broadcasts Commentariescited by 57
  30. 2021Bridge to Answer: Structure-aware Graph Interaction Network for Video Question Answeringcited by 56
  31. 2024VCA: Video Curious Agent for Long Video Understandingcited by 56
  32. 2021Dynamic Visual Reasoning by Learning Differentiable Physics Models from Video and Languagecited by 55
  33. 2022MIST: Multi-modal Iterative Spatial-Temporal Transformer for Long-form Video Question Answeringcited by 55
  34. 2021Target Adaptive Context Aggregation for Video Scene Graph Generationcited by 52
  35. 2022From Representation to Reasoning: Towards both Evidence and Commonsense Reasoning for Video Question-Answeringcited by 51
  36. 2024Grounded Multi-Hop VideoQA in Long-Form Egocentric Videoscited by 45
  37. 2024MMWorld: Towards Multi-discipline Multi-faceted World Model Evaluation in Videoscited by 43
  38. 2024VideoWebArena: Evaluating Long Context Multimodal Agents with Video Understanding Web Taskscited by 43

50 most cited of 304 All 304 (JSON)