World2VLM: Distilling World Model Imagination into VLMs for Dynamic Spatial Reasoning
On the map: World models → World models for language agents
also in 2 more
Abstract (arXiv)
Vision-language models (VLMs) have shown strong performance on static visual understanding, yet they still struggle with dynamic spatial reasoning that requires imagining how scenes evolve under egocentric motion. Recent efforts address this limitation either by scaling spatial supervision with synthetic data or by coupling VLMs with world models at inference time. However, the former often lacks explicit modeling of motion-conditioned state transitions, while the latter incurs substantial computational overhead. In this work, we propose World2VLM, a training framework that distills spatial imagination from a generative world model into a vision-language model. Given an initial observation and a parameterized camera trajectory, we use a view-consistent world model to synthesize geometrically aligned future views and derive structured supervision for both forward (action-to-outcome) and inverse (outcome-to-action) spatial reasoning. We post-train the VLM with a two-stage recipe on a compact dataset generated by this pipeline and evaluate it on multiple spatial reasoning benchmarks. World2VLM delivers consistent improvements over the base model across diverse benchmarks, including SAT-Real, SAT-Synthesized, VSI-Bench, and MindCube. It also outperforms the test-time world-model-coupled methods while eliminating the need for expensive inference-time generation. Our results suggest that world models can serve not only as inference-time tools, but also as effective training-time teachers, enabling VLMs to internalize spatial imagination in a scalable and efficient manner.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in World models for language agents, one of 11 directions in World models. Builds on 56 of its 62 references that are in the map. First posted on arXiv 29 Apr 2026.
Builds on
56 of 62 references.
Closest ancestors · 1
Same direction, newest first
Other references · 55
- 2024GPT-4o System Cardcited by 4,467
- 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
- 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
- 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,769
- 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,597
- 2024WorldSimBench: Towards Video Generation Models as World Simulatorscited by 867
- 2024Genie: Generative Interactive Environmentscited by 706
- 2024SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilitiescited by 679
- 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spacescited by 580
- 2024BLINK: Multimodal Large Language Models Can See but Not Perceivecited by 529
- 2025GEN3C: 3D-Informed World-Consistent Video Generation with Precise Camera Controlcited by 325
- 2023What's "up" with vision-language models? Investigating their struggle with spatial reasoningcited by 245
38 more
- 2023Parameter-Efficient Fine-Tuning Methods for Pretrained Language Models: A Critical Review and Assessmentcited by 227
- 2025Imagine while Reasoning in Space: Multimodal Visualization-of-Thoughtcited by 204
- 2025Stable Virtual Camera: Generative View Synthesis with Diffusion Modelscited by 179
- 2025MMSI-Bench: A Benchmark for Multi-Image Spatial Intelligencecited by 169
- 2023ManipLLM: Embodied Multimodal Large Language Model for Object-Centric Robotic Manipulationcited by 157
- 2024Video-of-Thought: Step-by-Step Video Reasoning from Perception to Cognitioncited by 151
- 20243DSRBench: A Comprehensive 3D Spatial Reasoning Benchmarkcited by 133
- 2025Reinforcing Spatial Reasoning in Vision-Language Models with Interwoven Thinking and Visual Drawingcited by 130
- 2025From Flatland to Space: Teaching Vision-Language Models to Perceive and Reason in 3Dcited by 127
- 2025Cambrian-S: Towards Spatial Supersensing in Videocited by 123
- 2024SAT: Dynamic Spatial Aptitude Training for Multimodal Language Modelscited by 111
- 2025OmniSpatial: Towards Comprehensive Spatial Reasoning Benchmark for Vision Language Modelscited by 105
- 2024EmbSpatial-Bench: Benchmarking Spatial Understanding for Embodied Tasks with Large Vision-Language Modelscited by 100
- 2025Why Is Spatial Reasoning Hard for VLMs? An Attention Mechanism Perspective on Focus Areascited by 93
- 2025SpatialLadder: Progressive Training for Spatial Reasoning in Vision-Language Modelscited by 83
- 2025MM-Spatial: Exploring 3D Spatial Understanding in Multimodal LLMscited by 72
- 2025SpatialReasoner: Towards Explicit and Generalizable 3D Spatial Reasoningcited by 65
- 2025SpatialCoT: Advancing Spatial Reasoning through Coordinate Alignment and Chain-of-Thought for Embodied Task Planningcited by 62
- 2025Spatial Reasoning with Vision-Language Models in Ego-Centric Multi-View Scenescited by 62
- 2024MMIU: Multimodal Multi-image Understanding for Evaluating Large Vision-Language Modelscited by 61
- 2025Grounded Chain-of-Thought for Multimodal Large Language Modelscited by 57
- 2025EmbodiedEval: Evaluate Multimodal LLMs as Embodied Agentscited by 55
- 2025How Far are VLMs from Visual Spatial Intelligence? A Benchmark-Driven Perspectivecited by 41
- 2026NeoVerse: Enhancing 4D World Model with in-the-wild Monocular Videoscited by 39
- 2025Beyond Semantics: Rediscovering Spatial Awareness in Vision-Language Modelscited by 35
- 2025Multimodal Spatial Reasoning in the Large Model Era: A Survey and Benchmarkscited by 31
- 2025Why Do MLLMs Struggle with Spatial Understanding? A Systematic Analysis from Data to Architecturecited by 30
- 2023MLLM-Bench: Evaluating Multimodal LLMs with Per-sample Criteriacited by 27
- 2025SITE: towards Spatial Intelligence Thorough Evaluationcited by 26
- 2026LiveWorld: Simulating Out-of-Sight Dynamics in Generative Video World Modelscited by 20
- 2025SpaRE: Enhancing Spatial Reasoning in Vision-Language Models with Synthetic Datacited by 19
- 2025Thinking with Camera: A Unified Multimodal Model for Camera-Centric Understanding and Generationcited by 19
- 2023Towards Grounded Visual Spatial Reasoning in Multi-Modal Vision Language Modelscited by 18
- 2026Walk through Paintings: Egocentric World Models from Internet Priorscited by 13
- 2026AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memoriescited by 13
- 2025CookBench: A Long-Horizon Embodied Planning Benchmark for Complex Cooking Scenarioscited by 10
- 2025Actial: Activate Spatial Reasoning Ability of Multimodal Large Language Modelscited by 10
- 2025SpatialDreamer: Incentivizing Spatial Reasoning via Active Mental Imagerycited by 10
50 most cited of 55 All 55 (JSON)
Built on by
Citing papers per year: 2026: 3.
3 papers.
Similar, not cited
Close in wording, no citation either way
- 2026WM-VLM: Probing Internal World Models for Interleaved Visual-Textual Reasoning
- 2026Thinking with Imagination: Agentic Visual Spatial Reasoning with World Simulators
- 2025Imagine in Space: Exploring the Frontier of Spatial Intelligence and Reasoning Efficiency in Vision Language Models
- 2026GeoWorld-VLM: Geometry from World Models for Vision-Language Models
- 2026Spatial-Interactor: Learning Spatial Reasoning through Interaction with the Observable Physical World
- 2025Probing the effectiveness of World Models for Spatial Reasoning through Test-time Scaling
- 2026Linear Mechanisms for Spatiotemporal Reasoning in Vision Language Models
- 2026Imaginative Perception Tokens Enhance Spatial Reasoning in Multimodal Language Models