mlsanityData to 7 Oct 2026

LVBench: An Extreme Long Video Understanding Benchmark

arXiv 2406.08035 ↗First posted 12 Jun 2024cs.CV cs.AIcited by 402

On the map: Multimodal LLMs / vision-language models → Video understanding with MLLMs

Abstract (arXiv)

Recent progress in multimodal large language models has markedly enhanced the understanding of short videos (typically under one minute), and several evaluation datasets have emerged accordingly. However, these advancements fall short of meeting the demands of real-world applications such as embodied intelligence for long-term decision-making, in-depth movie reviews and discussions, and live sports commentary, all of which require comprehension of long videos spanning several hours. To address this gap, we introduce LVBench, a benchmark specifically designed for long video understanding. Our dataset comprises publicly sourced videos and encompasses a diverse set of tasks aimed at long video comprehension and information extraction. LVBench is designed to challenge multimodal models to demonstrate long-term memory and extended comprehension capabilities. Our extensive evaluations reveal that current multimodal models still underperform on these demanding long video understanding tasks. Through LVBench, we aim to spur the development of more advanced models capable of tackling the complexities of long video comprehension. Our data and code are publicly available at: https://lvbench.github.io.

Key facts. Cited by 402 papers in the map (data through 7 Oct 2026). Placed in Video understanding with MLLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 31 of its 53 references that are in the map. First posted on arXiv 12 Jun 2024.

Builds on

31 of 53 references.

Closest ancestors · 6

Same direction, newest first

  1. 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis31 May
  2. 2024PLLaVA : Parameter-free LLaVA Extension from Images to Videos for Video Dense Captioning25 Apr
  3. 2023TimeChat: A Time-sensitive Multimodal Large Language Model for Long Video Understanding4 Dec
  4. 2023LLaMA-VID: An Image is Worth 2 Tokens in Large Language Models28 Nov
  5. 2023MVBench: A Comprehensive Multi-modal Video Understanding Benchmark28 Nov
  6. 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understanding5 Jun

Other references · 25

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  4. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
  5. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,048
  6. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,105
  7. 2023Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Modelscited by 1,047
  8. 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,033
  9. 2021GLM: General Language Model Pretraining with Autoregressive Blank Infillingcited by 1,026
  10. 2024Are We on the Right Way for Evaluating Large Vision-Language Models?cited by 893
  11. 2023VideoChat: Chat-Centric Video Understandingcited by 851
  12. 2023EVA-CLIP: Improved Training Techniques for CLIP at Scalecited by 786
13 more
  1. 2021NExT-QA:Next Phase of Question-Answering to Explaining Temporal Actionscited by 735
  2. 2023CogVLM: Visual Expert for Pretrained Language Modelscited by 704
  3. 2023CogAgent: A Visual Language Model for GUI Agentscited by 692
  4. 2023EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understandingcited by 666
  5. 2019ActivityNet-QA: A Dataset for Understanding Complex Web Videos via Question Answeringcited by 559
  6. 2023MovieChat: From Dense Token to Sparse Memory for Long Video Understandingcited by 536
  7. 2017TGIF-QA: Toward Spatio-Temporal Reasoning in Visual Question Answeringcited by 378
  8. 2020HERO: Hierarchical Encoder for Video+Language Omni-representation Pre-trainingcited by 362
  9. 2023Perception Test: A Diagnostic Benchmark for Multimodal Video Modelscited by 310
  10. 2015MovieQA: Understanding Stories in Movies through Question-Answeringcited by 304
  11. 2024ST-LLM: Large Language Models Are Effective Temporal Learnerscited by 126
  12. 2024CinePile: A Long Video Question Answering Dataset and Benchmarkcited by 112
  13. 2023SEED-Bench-2: Benchmarking Multimodal Large Language Modelscited by 96

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 27; 2025: 155; 2026: 220.

402 papers.

Sort:

By area:

  1. 2025Qwen3-VL Technical Reportcited by 2,534
  2. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,419
  3. 2025VideoLLaMA 3: Frontier Multimodal Foundation Models for Image and Video Understandingcited by 502
  4. 2025Qwen3-Omni Technical Reportcited by 451
  5. 2026Kimi K2.5: Visual Agentic Intelligencecited by 449
  6. 2025GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learningcited by 439
  7. 2024CogVLM2: Visual Language Models for Image and Video Understandingcited by 214
  8. 2024VideoChat-Flash: Hierarchical Compression for Long-Context Video Modelingcited by 212
  9. 2025InternVideo2.5: Empowering Video MLLMs with Long and Rich Context Modelingcited by 203
  10. 2025MiniCPM-V 4.5: Cooking Efficient MLLMs via Architecture, Data, and Training Recipecited by 174
  11. 2025WorldSense: Evaluating Real-world Omnimodal Understanding for Multimodal LLMscited by 147
  12. 2026Qwen3.5-Omni Technical Reportcited by 146
38 more
  1. 2024StreamingBench: Assessing the Gap for MLLMs to Achieve Streaming Video Understandingcited by 135
  2. 2025OVO-Bench: How Far is Your Video-LLMs from Real-World Online Video Understanding?cited by 134
  3. 2024Kangaroo: A Powerful Video-Language Model Supporting Long-context Video Inputcited by 132
  4. 2025MMVU: Measuring Expert-Level Multi-Discipline Video Understandingcited by 131
  5. 2025Deep Video Discovery: Agentic Search with Tool Use for Long-form Video Understandingcited by 94
  6. 2025StreamingVLM: Real-Time Understanding for Infinite Video Streamscited by 89
  7. 2024Apollo: An Exploration of Video Understanding in Large Multimodal Modelscited by 87
  8. 2025EgoLife: Towards Egocentric Life Assistantcited by 86
  9. 2024CG-Bench: Clue-grounded Question Answering Benchmark for Long Video Understandingcited by 82
  10. 2025T*: Re-thinking Temporal Search for Long-Form Video Understandingcited by 82
  11. 2025Flash-VStream: Efficient Real-Time Understanding for Long Video Streamscited by 78
  12. 2025VideoRAG: Retrieval-Augmented Generation with Extreme Long-Context Videoscited by 76
  13. 2025STI-Bench: Are MLLMs Ready for Precise Spatial-Temporal World Understanding?cited by 74
  14. 2024MME-Survey: A Comprehensive Survey on Evaluation of Multimodal LLMscited by 72
  15. 2024TimeMarker: A Versatile Video-LLM for Long and Short Video Understanding with Superior Temporal Localization Abilitycited by 69
  16. 2025MotionBench: Benchmarking and Improving Fine-grained Video Motion Understanding for Vision Language Modelscited by 69
  17. 2026MiniCPM-o 4.5: Towards Real-Time Full-Duplex Omni-Modal Interactioncited by 64
  18. 2025VideoChat-R1.5: Visual Test-Time Scaling to Reinforce Multimodal Reasoning by Iterative Perceptioncited by 60
  19. 2024Online Video Understanding: OVBench and VideoChat-Onlinecited by 58
  20. 2025LongVT: Incentivizing "Thinking with Long Videos" via Native Tool Callingcited by 58
  21. 2025SVBench: A Benchmark with Temporal Multi-Turn Dialogues for Streaming Video Understandingcited by 55
  22. 2024VCA: Video Curious Agent for Long Video Understandingcited by 54
  23. 2025ReAgent-V: A Reward-Driven Multi-Agent Framework for Video Understandingcited by 50
  24. 2025WorldMM: Dynamic Multimodal Memory Agent for Long Video Reasoningcited by 50
  25. 2026Seed1.8 Model Card: Towards Generalized Real-World Agencycited by 47
  26. 2026Video-MME-v2: Towards the Next Stage in Benchmarks for Comprehensive Video Understandingcited by 46
  27. 2025VideoMind: A Chain-of-LoRA Agent for Temporal-Grounded Video Reasoningcited by 41
  28. 2025VCR-Bench: A Comprehensive Evaluation Framework for Video Chain-of-Thought Reasoningcited by 41
  29. 2025The Amazon Nova Family of Models: Technical Report and Model Cardcited by 41
  30. 2025Long-VITA: Scaling Large Multi-modal Models to 1 Million Tokens with Leading Short-Context Accuracycited by 39
  31. 2025video-SALMONN 2: Caption-Enhanced Audio-Visual Large Language Modelscited by 39
  32. 2025FrameThinker: Learning to Think with Long Videos via Multi-Turn Frame Spotlightingcited by 39
  33. 2025A Survey of Multimodal Retrieval-Augmented Generationcited by 37
  34. 2025Video-MMLU: A Massive Multi-Discipline Lecture Understanding Benchmarkcited by 37
  35. 2024Lost in Time: A New Temporal Benchmark for VideoLLMscited by 36
  36. 2025Video-XL-2: Towards Very Long-Video Understanding Through Task-Aware KV Sparsificationcited by 36
  37. 2024AMEGO: Active Memory from long EGOcentric videoscited by 35
  38. 2025Vamba: Understanding Hour-Long Videos with Hybrid Mamba-Transformerscited by 35

50 most cited of 402 All 402 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024MMBench-Video: A Long-Form Multi-Shot Benchmark for Holistic Video Understanding
  2. 2024LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understanding
  3. 2024HumanVBench: Probing Human-Centric Video Understanding in MLLMs with Automatically Synthesized Benchmarks
  4. 2026MVPBench: A Multi-Video Perception Evaluation Benchmark for Multi-Modal Video Understanding
  5. 2025CFVBench: A Comprehensive Video Benchmark for Fine-grained Multimodal Retrieval-Augmented Generation
  6. 2024TemporalBench: Benchmarking Fine-grained Temporal Understanding for Multimodal Video Models
  7. 2024HourVideo: 1-Hour Video-Language Understanding
  8. 2024Visual Context Window Extension: A New Perspective for Long Video Understanding