mlsanityData to 7 Oct 2026

CinePile: A Long Video Question Answering Dataset and Benchmark

arXiv 2405.08813 ↗First posted 14 May 2024cs.CV cs.LG cs.MMcited by 112

Not on the map. Why?

Cited by 42 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Current datasets for long-form video understanding often fall short of providing genuine long-form comprehension challenges, as many tasks derived from these datasets can be successfully tackled by analyzing just one or a few random frames from a video. To address this issue, we present a novel dataset and benchmark, CinePile, specifically designed for authentic long-form video understanding. This paper details our innovative approach for creating a question-answer dataset, utilizing advanced LLMs with human-in-the-loop and building upon human-generated raw data. Our comprehensive dataset comprises 305,000 multiple-choice questions (MCQs), covering various visual and multimodal aspects, including temporal comprehension, understanding human-object interactions, and reasoning about events or actions within a scene. Additionally, we fine-tuned open-source Video-LLMs on the training split and evaluated both open-source and proprietary video-centric LLMs on the test split of our dataset. The findings indicate that although current models underperform compared to humans, fine-tuning these models can lead to significant improvements in their performance.

Key facts. Cited by 112 papers in the map (data through 7 Oct 2026). Builds on 46 of its 61 references that are in the map. First posted on arXiv 14 May 2024.

Builds on

46 of 61 references.

References · 46

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  4. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  5. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  6. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  7. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  8. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  9. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
  10. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
  11. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  12. 2014CIDEr: Consensus-based Image Description Evaluationcited by 2,027
34 more
  1. 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,385
  2. 2023MVBench: A Comprehensive Multi-modal Video Understanding Benchmarkcited by 1,086
  3. 2023Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Modelscited by 1,047
  4. 2023mPLUG-Owl: Modularization Empowers Large Language Models with Multimodalitycited by 1,038
  5. 2023Enhancing Chat Language Models by Scaling High-quality Instructional Conversationscited by 825
  6. 2016SPICE: Semantic Propositional Image Caption Evaluationcited by 820
  7. 2018Scaling Egocentric Vision: The EPIC-KITCHENS Datasetcited by 789
  8. 2021NExT-QA:Next Phase of Question-Answering to Explaining Temporal Actionscited by 735
  9. 2023EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understandingcited by 666
  10. 2024Self-Rewarding Language Modelscited by 654
  11. 2020Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillationcited by 591
  12. 2023Textbooks Are All You Need II: phi-1.5 technical reportcited by 559
  13. 2018From Recognition to Cognition: Visual Commonsense Reasoningcited by 553
  14. 2023Otter: A Multi-Modal Model with In-Context Instruction Tuningcited by 548
  15. 2023MovieChat: From Dense Token to Sparse Memory for Long Video Understandingcited by 536
  16. 2023InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and Generationcited by 488
  17. 2018TVQA: Localized, Compositional Video Question Answeringcited by 463
  18. 2017TGIF-QA: Toward Spatio-Temporal Reasoning in Visual Question Answeringcited by 378
  19. 2020HERO: Hierarchical Encoder for Video+Language Omni-representation Pre-trainingcited by 362
  20. 2023WhisperX: Time-Accurate Speech Transcription of Long-Form Audiocited by 320
  21. 2023Perception Test: A Diagnostic Benchmark for Multimodal Video Modelscited by 310
  22. 2015MovieQA: Understanding Stories in Movies through Question-Answeringcited by 304
  23. 2021Fake It Till You Make It: Face analysis in the wild using synthetic data alonecited by 200
  24. 2023A Simple LLM Framework for Long-Range Video Question-Answeringcited by 176
  25. 2023Simple synthetic data reduces sycophancy in large language modelscited by 174
  26. 2024Rephrasing the Web: A Recipe for Compute and Data-Efficient Language Modelingcited by 156
  27. 2021Towards Long-Form Video Understandingcited by 146
  28. 2021MAD: A Scalable Dataset for Language Grounding in Videos from Movie Audio Descriptionscited by 107
  29. 2023Video-Bench: A Comprehensive Benchmark and Toolkit for Evaluating Video-based Large Language Modelscited by 107
  30. 2020Condensed Movies: Story Based Retrieval with Contextual Embeddingscited by 95
  31. 2023Learning Vision from Models Rivals Learning Vision from Datacited by 71
  32. 2023AutoAD: Movie Description in Contextcited by 50
  33. 2023Feedback-guided Data Synthesis for Imbalanced Classificationcited by 33
  34. 2023PUG: Photorealistic and Semantically Controllable Synthetic Data for Representation Learningcited by 31

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 18; 2025: 64; 2026: 30.

112 papers.

Sort:

By area:

  1. 2025VideoLLaMA 3: Frontier Multimodal Foundation Models for Image and Video Understandingcited by 502
  2. 2024LVBench: An Extreme Long Video Understanding Benchmarkcited by 402
  3. 2024VideoChat-Flash: Hierarchical Compression for Long-Context Video Modelingcited by 212
  4. 2024Video-XL: Extra-Long Vision Language Model for Hour-Scale Video Understandingcited by 210
  5. 2025InternVideo2.5: Empowering Video MLLMs with Long and Rich Context Modelingcited by 203
  6. 2024Oryx MLLM: On-Demand Spatial-Temporal Understanding at Arbitrary Resolutioncited by 157
  7. 2024AuroraCap: Efficient, Performant Video Detailed Captioning and a New Benchmarkcited by 137
  8. 2024Kangaroo: A Powerful Video-Language Model Supporting Long-context Video Inputcited by 132
  9. 2026Molmo2: Open Weights and Data for Vision-Language Models with Video Understanding and Groundingcited by 132
  10. 2025EgoLife: Towards Egocentric Life Assistantcited by 86
  11. 2024CG-Bench: Clue-grounded Question Answering Benchmark for Long Video Understandingcited by 82
  12. 2025T*: Re-thinking Temporal Search for Long-Form Video Understandingcited by 82
38 more
  1. 2025Ola: Pushing the Frontiers of Omni-Modal Language Modelcited by 79
  2. 2024TOMATO: Assessing Visual Temporal Reasoning Capabilities in Multimodal Foundation Modelscited by 71
  3. 2024Too Many Frames, Not All Useful: Efficient Strategies for Long-Form Video QAcited by 68
  4. 2025Eagle 2.5: Boosting Long-Context Post-Training for Frontier Vision-Language Modelscited by 62
  5. 2024VCA: Video Curious Agent for Long Video Understandingcited by 54
  6. 2025Video-XL-Pro: Reconstructive Token Compression for Extremely Long Video Understandingcited by 49
  7. 2024VideoWebArena: Evaluating Long Context Multimodal Agents with Video Understanding Web Taskscited by 43
  8. 2025video-SALMONN 2: Caption-Enhanced Audio-Visual Large Language Modelscited by 39
  9. 2024Lost in Time: A New Temporal Benchmark for VideoLLMscited by 36
  10. 2024AMEGO: Active Memory from long EGOcentric videoscited by 35
  11. 2025CausalVQA: A Physically Grounded Causal Reasoning Benchmark for Video Modelscited by 35
  12. 2025Scaling Video-Language Models to 10K Frames via Hierarchical Differential Distillationcited by 33
  13. 2025Mavors: Multi-granularity Video Representation for Multimodal Large Language Modelcited by 31
  14. 2025MINERVA: Evaluating Complex Video Reasoningcited by 31
  15. 2025SlowFast-LLaVA-1.5: A Family of Token-Efficient Video Large Language Models for Long-Form Video Understandingcited by 30
  16. 2025A Shortcut-aware Video-QA Benchmark for Physical Understanding via Minimal Video Pairscited by 30
  17. 2025AVoCaDO: An Audiovisual Video Captioner Driven by Temporal Orchestrationcited by 26
  18. 2024Neptune: The Long Orbit to Benchmarking Long Video Understandingcited by 24
  19. 2025VRBench: A Benchmark for Multi-Step Reasoning in Long Narrative Videoscited by 24
  20. 2025BIMBA: Selective-Scan Compression for Long-Range Video Question Answeringcited by 23
  21. 2025Uni-MoE-2.0-Omni: Scaling Language-Centric Omnimodal Large Model with Advanced MoE, Training and Datacited by 21
  22. 2025Does Your Vision-Language Model Get Lost in the Long Video Sampling Dilemma?cited by 19
  23. 2025UniversalRAG: Retrieval-Augmented Generation over Corpora of Diverse Modalities and Granularitiescited by 18
  24. 2025SIV-Bench: A Video Benchmark for Social Interaction Understanding and Reasoningcited by 18
  25. 2025TimeSearch-R: Adaptive Temporal Search for Long-Form Video Understanding via Self-Verification Reinforcement Learningcited by 18
  26. 2025Memory-enhanced Retrieval Augmentation for Long Video Understandingcited by 17
  27. 2025Vid-SME: Membership Inference Attacks against Large Video Understanding Modelscited by 17
  28. 2025ALLVB: All-in-One Long Video Understanding Benchmarkcited by 16
  29. 2025ARGUS: Hallucination and Omission Evaluation in Video-LLMscited by 16
  30. 2025ReFoCUS: Reinforcement-guided Frame Optimization for Contextual Understandingcited by 15
  31. 2025X-LeBench: A Benchmark for Extremely Long Egocentric Video Understandingcited by 14
  32. 2025Vidi: Large Multimodal Models for Video Understanding and Editingcited by 14
  33. 2025SiLVR: A Simple Language-based Video Reasoning Frameworkcited by 14
  34. 2025LongViTU: Instruction Tuning for Long-Form Video Understandingcited by 12
  35. 2026DiaDem: Advancing Dialogue Descriptions in Audiovisual Video Captioning for Multimodal Large Language Modelscited by 12
  36. 2024Long Story Short: Story-level Video Understanding from 20K Short Filmscited by 11
  37. 2025MAGNET: A Multi-agent Framework for Finding Audio-Visual Needles by Reasoning over Multi-Video Haystackscited by 11
  38. 2025FindingDory: A Benchmark to Evaluate Memory in Embodied Agentscited by 11

50 most cited of 112 All 112 (JSON)