mlsanityData to 7 Oct 2026

How Much 3D Do Video Foundation Models Encode?

arXiv 2512.19949 ↗First posted 23 Dec 2025cs.CV cs.AIcited by 9

Not on the map. Why?

Abstract (arXiv)

Videos are continuous 2D projections of 3D worlds. After training on large video data, will global 3D understanding naturally emerge? We study this by quantifying the 3D understanding of existing Video Foundation Models (VidFMs) pretrained on vast video data. We propose the first model-agnostic framework that measures the 3D awareness of various VidFMs by estimating multiple 3D properties from their features via shallow read-outs. Our study presents meaningful findings regarding the 3D awareness of VidFMs on multiple axes. In particular, we show that state-of-the-art video generation models exhibit a strong understanding of 3D objects and scenes, despite not being trained on any 3D data. Such understanding can even surpass that of large expert models specifically trained for 3D tasks. Our findings, together with the 3D benchmarking of major VidFMs, provide valuable observations for building scalable 3D models.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 59 of its 66 references that are in the map. First posted on arXiv 23 Dec 2025.

Builds on

59 of 66 references.

References · 59

  1. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  2. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,536
  3. 2022Emergent Abilities of Large Language Modelscited by 2,483
  4. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,198
  5. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,755
  6. 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,597
  7. 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,526
  8. 2025VGGT: Visual Geometry Grounded Transformercited by 1,475
  9. 2022VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Trainingcited by 1,305
  10. 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,175
  11. 2022CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformerscited by 973
  12. 2024Grounding Image Matching in 3D with MASt3Rcited by 960
38 more
  1. 2019HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clipscited by 946
  2. 2021Common Objects in 3D: Large-Scale Learning and Evaluation of Real-life 3D Category Reconstructioncited by 646
  3. 2021VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text Understandingcited by 596
  4. 2024BLINK: Multimodal Large Language Models Can See but Not Perceivecited by 529
  5. 2023VideoMAE V2: Scaling Video Masked Autoencoders with Dual Maskingcited by 516
  6. 2023DL3DV-10K: A Large-Scale Scene Dataset for Deep Learning-based 3D Visioncited by 514
  7. 2024CameraCtrl: Enabling Camera Control for Text-to-Video Generationcited by 448
  8. 2023Emergent Correspondence from Image Diffusioncited by 437
  9. 2024ViewCrafter: Taming Video Diffusion Models for High-fidelity Novel View Synthesiscited by 392
  10. 2024Panda-70M: Captioning 70M Videos with Multiple Cross-Modality Teacherscited by 389
  11. 2025Fast3R: Towards 3D Reconstruction of 1000+ Images in One Forward Passcited by 356
  12. 2021Deep ViT Features as Dense Visual Descriptorscited by 327
  13. 2025GEN3C: 3D-Informed World-Consistent Video Generation with Precise Camera Controlcited by 325
  14. 2016YouTube-8M: A Large-Scale Video Classification Benchmarkcited by 324
  15. 2023A Tale of Two Features: Stable Diffusion Complements DINO for Zero-Shot Semantic Correspondencecited by 297
  16. 2025VBench-2.0: Advancing Video Generation Benchmark Suite for Intrinsic Faithfulnesscited by 289
  17. 2025Diffusion as Shader: 3D-aware Video Diffusion for Versatile Video Generation Controlcited by 184
  18. 2024Probing the 3D Awareness of Visual Foundation Modelscited by 175
  19. 2024CamCo: Camera-Controllable 3D-Consistent Image-to-Video Generationcited by 163
  20. 2017A Survey of Structure from Motioncited by 157
  21. 2025Open-Sora 2.0: Training a Commercial-Level Video Generation Model in 200kcited by 157
  22. 2024AC3D: Analyzing and Improving 3D Camera Control in Video Diffusion Transformerscited by 151
  23. 2025WorldScore: A Unified Evaluation Benchmark for World Generationcited by 143
  24. 2024ReconX: Reconstruct Any Scene from Sparse Views with Video Diffusion Modelcited by 139
  25. 2024DimensionX: Create Any 3D and 4D Scenes from a Single Image with Controllable Video Diffusioncited by 139
  26. 2024MV-DUSt3R+: Single-Stage Scene Reconstruction from Sparse Views In 2 Secondscited by 136
  27. 2025Video World Models with Long-term Spatial Memorycited by 124
  28. 2023Unsupervised Semantic Correspondence Using Stable Diffusioncited by 120
  29. 2024SAT: Dynamic Spatial Aptitude Training for Multimodal Language Modelscited by 111
  30. 2024Wonderland: Navigating 3D Scenes from a Single Imagecited by 98
  31. 2025CameraCtrl II: Dynamic Scene Exploration via Camera-controlled Video Diffusion Modelscited by 97
  32. 2025Aether: Geometric-Aware Unified World Modelingcited by 94
  33. 2025Voyager: Long-Range and World-Consistent Video Diffusion for Explorable 3D Scene Generationcited by 93
  34. 2024V3D: Video Diffusion Models are Effective 3D Generatorscited by 88
  35. 2025Geo4D: Leveraging Video Generators for Geometric 4D Scene Reconstructioncited by 66
  36. 2024World-consistent Video Diffusion with Explicit 3D Modelingcited by 62
  37. 2023Shadows Don't Lie and Lines Can't Bend! Generative Models don't know Projective Geometry...for nowcited by 61
  38. 2024Lexicon3D: Probing Visual Foundation Models for Complex 3D Scene Understandingcited by 46

50 most cited of 59 All 59 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 9.

9 papers.

  1. 2026WALL-WM: Carving World Action Modeling at the Event Jointscited by 14
  2. 2026World-R1: Reinforcing 3D Constraints for Text-to-Video Generationcited by 6
  3. 2026FrameCrafter: Novel View Synthesis as Video Completioncited by 3
  4. 2026EVA01: Unified Native 3D Understanding and Generation via Mixture-of-Transformerscited by 2
  5. 2026Know3D: Prompting 3D Generation with Knowledge from Vision-Language Modelscited by 1
  6. 2026SeeSE3: Emergence of 3D Space in Vision Featurescited by 1
  7. 2026GTASA: Ground Truth Annotations for Spatiotemporal Analysis, Evaluation and Training of Video Modelscited by 0
  8. 2026What, Where, and How: Probing Spatiotemporal Representations in Video Foundation Modelscited by 0
  9. 2026Rethinking Representations for World-Action Modelingcited by 0