mlsanityData to 7 Oct 2026

Video Understanding: From Geometry and Semantics to Unified Models

arXiv 2603.17840 ↗First posted 18 Mar 2026cs.CVcited by 5

Not on the map. Why?

Abstract (arXiv)

Video understanding aims to enable models to perceive, reason about, and interact with the dynamic visual world. In contrast to image understanding, video understanding inherently requires modeling temporal dynamics and evolving visual context, placing stronger demands on spatiotemporal reasoning and making it a foundational problem in computer vision. In this survey, we present a structured overview of video understanding by organizing the literature into three complementary perspectives: low-level video geometry understanding, high-level semantic understanding, and unified video understanding models. We further highlight a broader shift from isolated, task-specific pipelines toward unified modeling paradigms that can be adapted to diverse downstream objectives, enabling a more systematic view of recent progress. By consolidating these perspectives, this survey provides a coherent map of the evolving video understanding landscape, summarizes key modeling trends and design principles, and outlines open challenges toward building robust, scalable, and unified video foundation models.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 352 of its 404 references that are in the map. First posted on arXiv 18 Mar 2026.

Builds on

352 of 404 references.

References · 352

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2023GPT-4 Technical Reportcited by 18,132
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  5. 2025Qwen3 Technical Reportcited by 8,195
  6. 2023Visual Instruction Tuningcited by 7,792
  7. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  8. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
  9. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  10. 2025Qwen2.5-VL Technical Reportcited by 5,418
  11. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  12. 2024SAM 2: Segment Anything in Images and Videoscited by 2,784
38 more
  1. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
  2. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,536
  3. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
  4. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
  5. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,338
  6. 2022Video Diffusion Modelscited by 2,280
  7. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,198
  8. 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,597
  9. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,573
  10. 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,526
  11. 2025VGGT: Visual Geometry Grounded Transformercited by 1,475
  12. 2019SuperGlue: Learning Feature Matching with Graph Neural Networkscited by 1,437
  13. 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,385
  14. 2024Depth Anything V2cited by 1,385
  15. 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,375
  16. 2024Depth Anything: Unleashing the Power of Large-Scale Unlabeled Datacited by 1,346
  17. 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,173
  18. 2025SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Featurescited by 1,127
  19. 2015FlowNet: Learning Optical Flow with Convolutional Networkscited by 1,054
  20. 2023Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Modelscited by 1,047
  21. 2021LoFTR: Detector-Free Local Feature Matching with Transformerscited by 974
  22. 2024Grounding Image Matching in 3D with MASt3Rcited by 960
  23. 2017The 2017 DAVIS Challenge on Video Object Segmentationcited by 942
  24. 2025Emerging Properties in Unified Multimodal Pretrainingcited by 855
  25. 2025SAM 3: Segment Anything with Conceptscited by 835
  26. 2024Emu3: Next-Token Prediction is All You Needcited by 761
  27. 2021NExT-QA:Next Phase of Question-Answering to Explaining Temporal Actionscited by 735
  28. 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 734
  29. 2023VILA: On Pre-training for Visual Language Modelscited by 686
  30. 2023EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understandingcited by 666
  31. 2017Localizing Moments in Video with Natural Languagecited by 660
  32. 2024LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understandingcited by 644
  33. 2018Taskonomy: Disentangling Task Transfer Learningcited by 631
  34. 2025Depth Anything 3: Recovering the Visual Space from Any Viewscited by 626
  35. 2023NExT-GPT: Any-to-Any Multimodal LLMcited by 610
  36. 2023LightGlue: Local Feature Matching at Light Speedcited by 583
  37. 2017TALL: Temporal Activity Localization via Language Querycited by 581
  38. 2019ActivityNet-QA: A Dataset for Understanding Complex Web Videos via Question Answeringcited by 559

50 most cited of 352 All 352 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 5.

5 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026Video Understanding: Through A Temporal Lens
  2. 2026Video-Oasis: Rethinking Evaluation of Video Understanding
  3. 2025Video-LMM Post-Training: A Deep Dive into Video Reasoning with Large Multimodal Models
  4. 2024MMBench-Video: A Long-Form Multi-Shot Benchmark for Holistic Video Understanding
  5. 2025Video-CoT: A Comprehensive Dataset for Spatiotemporal Understanding of Videos Based on Chain-of-Thought
  6. 2024Video-of-Thought: Step-by-Step Video Reasoning from Perception to Cognition
  7. 2026VideoSeeker: Incentivizing Instance-level Video Understanding via Native Agentic Tool Invocation
  8. 2025VLM4D: Towards Spatiotemporal Awareness in Vision Language Models