# Which Pretraining Paradigm Better Serves Spatial Intelligence? An Empirical Comparison of Vision-Language and Video Generation Models

arXiv: https://arxiv.org/abs/2605.28132 · HTML: https://mlsanity.com/paper/2605.28132/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 25 of its 40 references that are in the map.
- First posted on arXiv 27 May 2026.

## Builds on (most cited first, 25 of 25)

- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [Wan: Open and Advanced Large-Scale Video Generative Models](https://mlsanity.com/paper/2503.20314/) (2025)
- [InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency](https://mlsanity.com/paper/2508.18265/) (2025)
- [SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilities](https://mlsanity.com/paper/2401.12168/) (2024)
- [Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spaces](https://mlsanity.com/paper/2412.14171/) (2024)
- [GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learning](https://mlsanity.com/paper/2507.01006/) (2025)
- [Object Goal Navigation using Goal-Oriented Semantic Exploration](https://mlsanity.com/paper/2007.00643/) (2020)
- [Asymmetric Loss For Multi-Label Classification](https://mlsanity.com/paper/2009.14119/) (2020)
- [Embodied Question Answering](https://mlsanity.com/paper/1711.11543/) (2017)
- [Hydra: A Real-time Spatial Perception System for 3D Scene Graph Construction and Optimization](https://mlsanity.com/paper/2201.13360/) (2022)
- [OpenDriveVLA: Towards End-to-end Autonomous Driving with Large Vision Language Action Model](https://mlsanity.com/paper/2503.23463/) (2025)
- [EmbodiedScan: A Holistic Multi-Modal 3D Perception Suite Towards Embodied AI](https://mlsanity.com/paper/2312.16170/) (2023)
- [StarVLA: A Lego-like Codebase for Vision-Language-Action Model Developing](https://mlsanity.com/paper/2604.05014/) (2026)
- [Does CLIP Bind Concepts? Probing Compositionality in Large Image Models](https://mlsanity.com/paper/2212.10537/) (2022)
- [Aether: Geometric-Aware Unified World Modeling](https://mlsanity.com/paper/2503.18945/) (2025)
- [Scaling Spatial Intelligence with Multimodal Foundation Models](https://mlsanity.com/paper/2511.13719/) (2025)
- [SpatialReasoner: Towards Explicit and Generalizable 3D Spatial Reasoning](https://mlsanity.com/paper/2504.20024/) (2025)
- [A Survey for Foundation Models in Autonomous Driving](https://mlsanity.com/paper/2402.01105/) (2024)
- [MetaSpatial: Reinforcing 3D Spatial Reasoning in VLMs for the Metaverse](https://mlsanity.com/paper/2503.18470/) (2025)
- [3DMIT: 3D Multi-modal Instruction Tuning for Scene Understanding](https://mlsanity.com/paper/2401.03201/) (2024)
- [SPA: 3D Spatial-Awareness Enables Effective Embodied Representation](https://mlsanity.com/paper/2410.08208/) (2024)
- [IGGT: Instance-Grounded Geometry Transformer for Semantic 3D Reconstruction](https://mlsanity.com/paper/2510.22706/) (2025)
- [NavBench: Probing Multimodal Large Language Models for Embodied Navigation](https://mlsanity.com/paper/2506.01031/) (2025)
- [BIP3D: Bridging 2D Images and 3D Perception for Embodied Intelligence](https://mlsanity.com/paper/2411.14869/) (2024)
- [DVD: Deterministic Video Depth Estimation with Generative Priors](https://mlsanity.com/paper/2603.12250/) (2026)

## Abstract (arXiv)

Spatial intelligence requires visual representations that capture both semantic objects and geometric structure in the physical world. To support this, two major pre-training schemes are now widely used as foundation backbones: Vision-Language Models (VLMs), which use language supervision to align visual observations with semantic concepts, and Video Generation Models (VGMs), which learn from temporally evolving visual worlds. However, it still remains unclear which pre-training scheme provides a better representation substrate for spatial intelligence. In this paper, we present the first systematic frozen-feature probing study of VLMs and VGMs across three representative axes of spatial intelligence: semantic tagging, instance grouping, and 3D geometry prediction. Using the lightweight probe, our framework enables a controlled comparison of what information is already encoded in frozen representations from two model families. Experimental results reveal a clear complementarity: VLMs are stronger at semantic tagging and instance grouping, while VGMs provide more accessible signals for dense geometry and camera motion. Moreover, a naive fusion of the two already yields a representation that excels at both geometry and semantics, suggesting a promising direction for building stronger spatial-intelligence backbones by effectively integrating features from both model families. Our code is available at hrefhttps://github.com/om-ai-lab/Probing-VLM-VGMhttps://github.com/om-ai-lab/Probing-VLM-VGM.