mlsanityData to 14 Oct 2026

SpatialVID: A Large-Scale Video Dataset with Spatial Annotations

arXiv 2509.09676 ↗First posted 11 Sep 2025cs.CVcited by 77

Not on the map. Why?

Cited by 33 in World models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Significant progress has been made in spatial intelligence, spanning both spatial reconstruction and world exploration. However, the scalability and real-world fidelity of current models remain severely constrained by the scarcity of large-scale, high-quality training data. While several datasets provide camera pose information, they are typically limited in scale, diversity, and annotation richness, particularly for real-world dynamic scenes with ground-truth camera motion. To this end, we collect SpatialVID, a dataset consists of a large corpus of in-the-wild videos with diverse scenes, camera movements and dense 3D annotations such as per-frame camera poses, depth, and motion instructions. Specifically, we collect more than 21,000 hours of raw videos, and process them into 2.7 million clips through a hierarchical filtering pipeline, totaling 7,089 hours of dynamic content. A subsequent annotation pipeline enriches these clips with detailed spatial and semantic information, including camera poses, depth maps, dynamic masks, structured captions, and serialized motion instructions. Analysis of SpatialVID's data statistics reveals a richness and diversity that directly fosters improved model generalization and performance, establishing it as a key asset for the video and 3D vision research community.

Key facts. Cited by 77 papers in the map (data through 14 Oct 2026). Builds on 79 of its 90 references that are in the map. First posted on arXiv 11 Sep 2025.

Builds on

79 of 90 references.

References · 79

  1. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  2. 2025Qwen3 Technical Reportcited by 8,820
  3. 2020Scaling Laws for Neural Language Modelscited by 6,586
  4. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
  5. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,117
  6. 2024SAM 2: Segment Anything in Images and Videoscited by 2,863
  7. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,708
  8. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,581
  9. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,451
  10. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,261
  11. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,822
  12. 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,660
38 more
  1. 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,567
  2. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,540
  3. 2025VGGT: Visual Geometry Grounded Transformercited by 1,539
  4. 2024Depth Anything V2cited by 1,422
  5. 2024Depth Anything: Unleashing the Power of Large-Scale Unlabeled Datacited by 1,360
  6. 2015ORB-SLAM: a Versatile and Accurate Monocular SLAM Systemcited by 988
  7. 2024Grounding Image Matching in 3D with MASt3Rcited by 983
  8. 2025Cosmos World Foundation Model Platform for Physical AIcited by 864
  9. 2023LRM: Large Reconstruction Model for Single Image to 3Dcited by 849
  10. 2018MVSNet: Depth Inference for Unstructured Multi-view Stereocited by 788
  11. 2024LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creationcited by 739
  12. 2021DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Camerascited by 687
  13. 2021Common Objects in 3D: Large-Scale Learning and Evaluation of Real-life 3D Category Reconstructioncited by 652
  14. 2020Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understandingcited by 604
  15. 2023MotionCtrl: A Unified and Flexible Motion Controller for Video Generationcited by 580
  16. 2025Continuous 3D Perception Model with Persistent Statecited by 548
  17. 2023DL3DV-10K: A Large-Scale Scene Dataset for Deep Learning-based 3D Visioncited by 537
  18. 2024Revisiting Feature Prediction for Learning Visual Representations from Videocited by 531
  19. 2020TartanAir: A Dataset to Push the Limits of Visual SLAMcited by 506
  20. 2019BlendedMVS: A Large-scale Dataset for Generalized Multi-view Stereo Networkscited by 490
  21. 2024CameraCtrl: Enabling Camera Control for Text-to-Video Generationcited by 460
  22. 2024MonST3R: A Simple Approach for Estimating Geometry in the Presence of Motioncited by 442
  23. 2019VATEX: A Large-Scale, High-Quality Multilingual Dataset for Video-and-Language Researchcited by 414
  24. 2024ViewCrafter: Taming Video Diffusion Models for High-fidelity Novel View Synthesiscited by 403
  25. 2024Panda-70M: Captioning 70M Videos with Multiple Cross-Modality Teacherscited by 393
  26. 2024UniDepth: Universal Monocular Metric Depth Estimationcited by 382
  27. 2025Fast3R: Towards 3D Reconstruction of 1000+ Images in One Forward Passcited by 367
  28. 2020Virtual KITTI 2cited by 353
  29. 2024GS-LRM: Large Reconstruction Model for 3D Gaussian Splattingcited by 349
  30. 2024OpenVid-1M: A Large-Scale High-Quality Dataset for Text-to-video Generationcited by 341
  31. 2025ReCamMaster: Camera-Controlled Generative Rendering from A Single Videocited by 287
  32. 2023PointOdyssey: A Large-Scale Synthetic Dataset for Long-Term Point Trackingcited by 276
  33. 2023DragNUWA: Fine-grained Control in Video Generation by Integrating Text, Image, and Trajectorycited by 264
  34. 2024MegaSaM: Accurate, Fast, and Robust Structure and Motion from Casual Dynamic Videoscited by 228
  35. 2023MVImgNet: A Large-scale Dataset of Multi-view Imagescited by 220
  36. 2022Monocular Dynamic View Synthesis: A Reality Checkcited by 200
  37. 2024LVSM: A Large View Synthesis Model with Minimal 3D Inductive Biascited by 189
  38. 2025UniDepthV2: Universal Monocular Metric Depth Estimation Made Simplercited by 173

50 most cited of 79 All 79 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 11; 2026: 66.

77 papers.

Sort:

By area:

  1. 2026Advancing Open-source World Modelscited by 154
  2. 2026Matrix-Game 3.0: Real-Time and Streaming Interactive World Model with Long-Horizon Memorycited by 100
  3. 2025RELIC: Interactive Video World Model with Long-Horizon Memorycited by 96
  4. 2026SANA-WM: Efficient Minute-Scale World Modeling with Hybrid Linear Diffusion Transformercited by 55
  5. 2026Infinite Worlds with Versatile Interactionscited by 48
  6. 2026NeoVerse: Enhancing 4D World Model with in-the-wild Monocular Videoscited by 46
  7. 2025Spatia: Video Generation with Updatable Spatial Memorycited by 40
  8. 2026DreamX-World 1.0: A General-Purpose Interactive World Modelcited by 40
  9. 2026LingBot-Map: Geometric Context Transformer for Streaming 3D Reconstructioncited by 37
  10. 2025Astra: General Interactive World Model with Autoregressive Denoisingcited by 33
  11. 2026VerseCrafter: Dynamic Realistic Video World Model with 4D Geometric Controlcited by 28
  12. 2026LiveWorld: Simulating Out-of-Sight Dynamics in Generative Video World Modelscited by 28
38 more
  1. 2026VGGT-Omegacited by 26
  2. 2025E-RayZer: Self-supervised 3D Reconstruction as Spatial Visual Pre-trainingcited by 22
  3. 2026minWM: A Full-Stack Open-Source Framework for Real-Time Interactive Video World Modelscited by 19
  4. 2026MBench: A Comprehensive Benchmark on Memory Capability for Video World Modelscited by 18
  5. 2026iWorld-Bench: A Benchmark for Interactive World Models with a Unified Action Generation Frameworkcited by 14
  6. 2026WildWorld: A Large-Scale Dataset for Dynamic World Modeling with Actions and Explicit State toward Generative ARPGcited by 13
  7. 2026SolarWM: Open Data and Scalable Training for Long-Horizon Video World Modelscited by 12
  8. 2026LoMa: Local Feature Matching Revisitedcited by 11
  9. 2026AlayaWorld: Interactive Long-Horizon World Modeling -- Full Technical Reportcited by 11
  10. 2025Pelican-VL 1.0: A Foundation Brain Model for Embodied Intelligencecited by 10
  11. 20254D-VGGT: A General Foundation Model with SpatioTemporal Awareness for Dynamic Scene Geometry Estimationcited by 10
  12. 2025LongVie 2: Multimodal Controllable Ultra-Long Video World Modelcited by 9
  13. 2026Flow3r: Factored Flow Prediction for Scalable Visual Geometry Learningcited by 9
  14. 2026UCM: Unified Modeling of Camera Control and Memory with Time-aware Positional Encoding Warping for World Modelscited by 9
  15. 2026Aurora: Unified Video Editing with a Tool-Using Agentcited by 9
  16. 2026MemLearner: Learning to Query Context memory for Video World Modelscited by 9
  17. 2025One4D: Unified 4D Generation and Reconstruction via Decoupled LoRA Controlcited by 8
  18. 2026VideoWeave: Unlocking Geometric Consistency in Video Generation via Joint Geometry-Video Modelingcited by 7
  19. 2026Compression and Retrieval: Implicit Memory Retrieval for Video World Modelscited by 6
  20. 2025MASS: Motion-Aware Spatial-Temporal Grounding for Physics Reasoning and Comprehension in Vision-Language Modelscited by 5
  21. 2026ArtifactWorld: Scaling 3D Gaussian Splatting Artifact Restoration via Video Generation Modelscited by 5
  22. 2026Worldscape-MoE: A Unified Mixture-of-Experts World Model for Scalable Heterogeneous Action Controlcited by 5
  23. 20264DThinker: Thinking with 4D Imagery for Dynamic Spatial Understandingcited by 4
  24. 2026Zing-0.5: Toward Playable Worlds with Real-Time Joint Action and Text Controlcited by 4
  25. 2025Bridging VLMs and Embodied Intelligence with Deliberate Practice Policy Optimizationcited by 3
  26. 2026Geometry-Guided Camera Motion Understanding in VideoLLMscited by 3
  27. 2026Building a Precise Video Language with Human-AI Oversightcited by 3
  28. 2026Holo-World: Unified Camera, Object and Weather Control for Video World Modelcited by 3
  29. 2026Sekai2: From World Exploration to Interactive World Modelingcited by 3
  30. 2025GimbalDiffusion: Gravity-Aware Camera Control for Video Generationcited by 2
  31. 2026What if? Emulative Simulation with World Models for Situated Reasoningcited by 2
  32. 2026Mask Forcing: Improving Autoregressive Video Diffusion Distillation via Dual-Noise Masking Rolloutcited by 2
  33. 2026SceneScribe-1M: A Large-Scale Video Dataset with Comprehensive Geometric and Semantic Annotationscited by 1
  34. 2026Who Handles Orientation? Investigating Invariance in Feature Matchingcited by 1
  35. 2026Relit-LiVE: Relight Video by Jointly Learning Environment Videocited by 1
  36. 2026Uncovering and Shaping the Latent Representation of 3D Scene Topology in Vision-Language Modelscited by 1
  37. 20264DP-QA: Scalable QA for 4D Perception in Vision Language Modelscited by 1
  38. 2026MotionAtlas: Detailed Region Captioning for Motion-Centric Videoscited by 1

50 most cited of 77 All 77 (JSON)