mlsanityData to 14 Oct 2026

OVO-Bench: How Far is Your Video-LLMs from Real-World Online Video Understanding?

arXiv 2501.05510 ↗First posted 9 Jan 2025cs.CV cs.AIcited by 142

Not on the map. Why?

Cited by 43 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Temporal Awareness, the ability to reason dynamically based on the timestamp when a question is raised, is the key distinction between offline and online video LLMs. Unlike offline models, which rely on complete videos for static, post hoc analysis, online models process video streams incrementally and dynamically adapt their responses based on the timestamp at which the question is posed. Despite its significance, temporal awareness has not been adequately evaluated in existing benchmarks. To fill this gap, we present OVO-Bench (Online-VideO-Benchmark), a novel video benchmark that emphasizes the importance of timestamps for advanced online video understanding capability benchmarking. OVO-Bench evaluates the ability of video LLMs to reason and respond to events occurring at specific timestamps under three distinct scenarios: (1) Backward tracing: trace back to past events to answer the question. (2) Real-time understanding: understand and respond to events as they unfold at the current timestamp. (3) Forward active responding: delay the response until sufficient future information becomes available to answer the question accurately. OVO-Bench comprises 12 tasks, featuring 644 unique videos and approximately human-curated 2,800 fine-grained meta-annotations with precise timestamps. We combine automated generation pipelines with human curation. With these high-quality samples, we further developed an evaluation pipeline to systematically query video LLMs along the video timeline. Evaluations of nine Video-LLMs reveal that, despite advancements on traditional benchmarks, current models struggle with online video understanding, showing a significant gap compared to human agents. We hope OVO-Bench will drive progress in video LLMs and inspire future research in online video reasoning. Our benchmark and code can be accessed at https://github.com/JoeLeelyf/OVO-Bench.

Key facts. Cited by 142 papers in the map (data through 14 Oct 2026). Builds on 48 of its 64 references that are in the map. First posted on arXiv 9 Jan 2025.

Builds on

48 of 64 references.

References · 48

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  4. 2024GPT-4o System Cardcited by 4,588
  5. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  6. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  7. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,876
  8. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
  9. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
  10. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,556
  11. 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understandingcited by 1,471
  12. 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,413
36 more
  1. 2023MVBench: A Comprehensive Multi-modal Video Understanding Benchmarkcited by 1,109
  2. 2023Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Modelscited by 1,062
  3. 2024Are We on the Right Way for Evaluating Large Vision-Language Models?cited by 914
  4. 2023VideoChat: Chat-Centric Video Understandingcited by 857
  5. 2021NExT-QA:Next Phase of Question-Answering to Explaining Temporal Actionscited by 744
  6. 2023EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understandingcited by 680
  7. 2024LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understandingcited by 665
  8. 2017Towards Automatic Learning of Procedures from Web Instructional Videoscited by 619
  9. 2019ActivityNet-QA: A Dataset for Understanding Complex Web Videos via Question Answeringcited by 561
  10. 2023LLaMA-VID: An Image is Worth 2 Tokens in Large Language Modelscited by 546
  11. 2023MovieChat: From Dense Token to Sparse Memory for Long Video Understandingcited by 542
  12. 2024LVBench: An Extreme Long Video Understanding Benchmarkcited by 410
  13. 2017TGIF-QA: Toward Spatio-Temporal Reasoning in Visual Question Answeringcited by 378
  14. 2023Chat-UniVi: Unified Visual Representation Empowers Large Language Models with Image and Video Understandingcited by 371
  15. 2024LongVU: Spatiotemporal Adaptive Compression for Long Video-Language Understandingcited by 356
  16. 2019COIN: A Large-scale Dataset for Comprehensive Instructional Video Analysiscited by 318
  17. 2023Perception Test: A Diagnostic Benchmark for Multimodal Video Modelscited by 314
  18. 2023InternLM-XComposer: A Vision-Language Large Model for Advanced Text-image Comprehension and Compositioncited by 297
  19. 2024MA-LMM: Memory-Augmented Large Multimodal Model for Long-Term Video Understandingcited by 229
  20. 2024VideoLLM-online: Online Video Large Language Model for Streaming Videocited by 225
  21. 2024STAR: A Benchmark for Situated Reasoning in Real-World Videoscited by 213
  22. 2020MovieNet: A Holistic Dataset for Movie Understandingcited by 206
  23. 2019Cross-task weakly supervised learning from instructional videoscited by 199
  24. 2024MMBench-Video: A Long-Form Multi-Shot Benchmark for Holistic Video Understandingcited by 182
  25. 2024Flash-VStream: Memory-Based Real-Time Understanding for Long Video Streamscited by 169
  26. 2024Streaming Long Video Understanding with Large Language Modelscited by 167
  27. 2024StreamingBench: Assessing the Gap for MLLMs to Achieve Streaming Video Understandingcited by 144
  28. 2025Dispider: Enabling Video LLMs with Active Real-Time Interaction via Disentangled Perception, Decision, and Reactioncited by 127
  29. 2024ST-LLM: Large Language Models Are Effective Temporal Learnerscited by 126
  30. 2024HourVideo: 1-Hour Video-Language Understandingcited by 113
  31. 2024TemporalBench: Benchmarking Fine-grained Temporal Understanding for Multimodal Video Modelscited by 97
  32. 2023Hierarchical Video-Moment Retrieval and Step-Captioningcited by 83
  33. 2024E.T. Bench: Towards Open-Ended Event-Level Video-Language Understandingcited by 70
  34. 2024EventHallusion: Diagnosing Event Hallucinations in Video LLMscited by 57
  35. 2024InternLM-XComposer2.5-OmniLive: A Comprehensive Multimodal System for Long-term Streaming Video and Audio Interactionscited by 48
  36. 2023AutoEval-Video: An Automatic Benchmark for Assessing Large Vision Language Models in Open-Ended Video Question Answeringcited by 40

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 30; 2026: 112.

142 papers.

Sort:

By area:

  1. 2025TimeChat-Online: 80% Visual Tokens are Naturally Redundant in Streaming Videoscited by 97
  2. 2025Seeing, Listening, Remembering, and Reasoning: A Multimodal Agent with Long-Term Memorycited by 97
  3. 2025StreamingVLM: Real-Time Understanding for Infinite Video Streamscited by 95
  4. 2025StreamForest: Efficient Online Video Understanding with Persistent Event Memorycited by 78
  5. 2025StreamBridge: Turning Your Offline Video Large Language Model into a Proactive Streaming Assistantcited by 63
  6. 2025OmniMMI: A Comprehensive Multi-modal Interaction Benchmark in Streaming Video Contextscited by 51
  7. 2026Seed1.8 Model Card: Towards Generalized Real-World Agencycited by 50
  8. 2025ViSpeak: Visual Instruction Feedback in Streaming Videoscited by 42
  9. 2026HERMES: KV Cache as Hierarchical Memory for Efficient Streaming Video Understandingcited by 41
  10. 2025MMSI-Video-Bench: A Holistic Benchmark for Video-Based Spatial Intelligencecited by 38
  11. 2025Accelerating Streaming Video Large Language Models via Hierarchical Token Compressioncited by 37
  12. 2025StreamAgent: Towards Anticipatory Agents for Streaming Video Understandingcited by 33
38 more
  1. 2026A Simple Baseline for Streaming Video Understandingcited by 33
  2. 2026FluxMem: Adaptive Hierarchical Memory for Streaming Video Understandingcited by 28
  3. 2025ProactiveVideoQA: A Comprehensive Benchmark Evaluating Proactive Interactions in Video Large Language Modelscited by 27
  4. 2025Streaming Video Instruction Tuningcited by 26
  5. 2025LiveStar: Live Streaming Assistant for Real-World Online Video Understandingcited by 25
  6. 2025Ming-Flash-Omni: A Sparse, Unified Architecture for Multimodal Perception and Generationcited by 22
  7. 2026Video Streaming Thinking: VideoLLMs Can Watch and Think Simultaneouslycited by 22
  8. 2025ST-Think: How Multimodal Large Language Models Reason About 4D Worlds from Ego-Centric Videoscited by 21
  9. 2025RTV-Bench: Benchmarking MLLM Continuous Perception, Understanding and Reasoning through Real-Time Videocited by 21
  10. 2026StreamReady: Learning What to Answer and When in Long Streaming Videoscited by 19
  11. 2026Thinking in Streaming Videocited by 16
  12. 2026AURA: Always-On Understanding and Real-Time Assistance via Video Streamscited by 16
  13. 2026RIVER: A Real-Time Interaction Benchmark for Video LLMscited by 14
  14. 2025StreamEQA: Towards Streaming Video Understanding for Embodied Scenarioscited by 12
  15. 2026EventMemAgent: Hierarchical Event-Centric Memory for Online Video Understanding with Adaptive Tool Usecited by 12
  16. 2025Native Visual Understanding: Resolving Resolution Dilemmas in Vision-Language Modelscited by 11
  17. 2025TeleEgo: Benchmarking Egocentric AI Assistants in the Wildcited by 11
  18. 2026PhoStream: Benchmarking Real-World Streaming for Omnimodal Assistants in Mobile Scenarioscited by 10
  19. 2026VideoChat3: Fully Open Video MLLM for Efficient and Generalist Video Understandingcited by 10
  20. 2026Mage-VL: An Efficient Codec-Native Streaming Multimodal Foundation Modelcited by 10
  21. 2025LiveCC: Learning Video LLM with Streaming Speech Transcription at Scalecited by 9
  22. 2026PEARL: Personalized Streaming Video Understanding Modelcited by 9
  23. 2026Scaling the Long Video Understanding of Multimodal Large Language Models via Visual Memory Mechanismcited by 9
  24. 2025Video-LMM Post-Training: A Deep Dive into Video Reasoning with Large Multimodal Modelscited by 8
  25. 2026Semantic-Aware Adaptive Visual Memory for Streaming Video Understandingcited by 8
  26. 2026Em-Garde: A Propose-Match Framework for Proactive Streaming Video Understandingcited by 7
  27. 2026IPIBench: Evaluating Interactive Proactive Intelligence of MLLMs under Continuous Streamscited by 7
  28. 2026What Should a Streaming Video Model Remember?cited by 7
  29. 2025AViLA: Asynchronous Vision-Language Agent for Streaming Multimodal Data Interactioncited by 6
  30. 2025DOCR-Inspector: Fine-Grained and Automated Evaluation of Document Parsing with VLMcited by 6
  31. 2026Event-VStream: Event-Driven Real-Time Understanding for Long Video Streamscited by 6
  32. 2026SVCBench: A Streaming Video Counting Benchmark for Spatial-Temporal State Maintenancecited by 6
  33. 2026OVO-S-Bench: A Hierarchical Benchmark for Streaming Spatial Intelligence in Multimodal LLMscited by 6
  34. 2026EgoSAT: A Comprehensive Benchmark of Egocentric Streaming Interaction Understandingcited by 6
  35. 2025Building Egocentric Procedural AI Assistant: Methods, Benchmarks, and Challengescited by 5
  36. 2025Know-Show: Benchmarking Video-Language Models on Spatio-Temporal Grounded Reasoningcited by 5
  37. 2026LifeEval: A Multimodal Benchmark for Assistive AI in Egocentric Daily Life Taskscited by 5
  38. 2026CurveStream: Boosting Streaming Video Understanding in MLLMs via Curvature-Aware Hierarchical Visual Memory Managementcited by 5

50 most cited of 142 All 142 (JSON)