Seeing through Imagination: Learning Scene Geometry via Implicit Spatial World Modeling
On the map: World models → World models for language agents
also in 2 more
Abstract (arXiv)
Spatial reasoning, the ability to understand and interpret the 3D structure of the world, is a critical yet underdeveloped capability in Multimodal Large Language Models (MLLMs). Current methods predominantly rely on verbal descriptive tuning, which suffers from visual illiteracy, i.e., they learn spatial concepts through textual symbols alone, devoid of connection to their visual manifestations. To bridge this gap, this paper introduces MILO, an Implicit spatIaL wOrld modeling paradigm that simulates human-like spatial imagination. MILO integrates a visual generator to provide geometry-aware feedback, thereby implicitly grounding the MLLM's symbolic reasoning in perceptual experience. Complementing this paradigm, we propose RePE (Relative Positional Encoding), a novel encoding scheme that captures relative camera-pose transformations, offering superior performance over absolute coordinate systems. To support the training, we construct GeoGen, a large-scale Geometry-aware Generative dataset with approximately 2,241 videos and 67,827 observation-action-outcome triplets. Experiments demonstrate that our approach significantly enhances spatial reasoning capabilities across multiple baselines and benchmarks, offering a more holistic understanding of 3D space.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in World models for language agents, one of 11 directions in World models. Builds on 72 of its 82 references that are in the map. First posted on arXiv 1 Dec 2025.
Builds on
72 of 82 references.
Closest ancestors · 1
Same direction, newest first
Other references · 71
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2025Qwen2.5-VL Technical Reportcited by 5,587
- 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
- 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,117
- 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
- 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,581
- 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,653
- 2025VGGT: Visual Geometry Grounded Transformercited by 1,539
- 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,050
- 2020Mastering Atari with Discrete World Modelscited by 1,007
- 2018Recurrent World Models Facilitate Policy Evolutioncited by 785
- 2023ScanNet++: A High-Fidelity Dataset of 3D Indoor Scenescited by 764
38 more
- 2024Genie: Generative Interactive Environmentscited by 747
- 2024SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilitiescited by 706
- 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spacescited by 611
- 2025Continuous 3D Perception Model with Persistent Statecited by 548
- 2023DriveLM: Driving with Graph Visual Question Answeringcited by 536
- 2025SpatialVLA: Exploring Spatial Representations for Visual-Language-Action Modelcited by 519
- 2025Video-R1: Reinforcing Video Reasoning in MLLMscited by 484
- 2019ScanRefer: 3D Object Localization in RGB-D Scans using Natural Languagecited by 452
- 2023An Embodied Generalist Agent in 3D Worldcited by 365
- 2021ScanQA: 3D Question Answering for Spatial Scene Understandingcited by 346
- 2022SQA3D: Situated Question Answering in 3D Scenescited by 314
- 2023LMDrive: Closed-Loop End-to-End Driving with Large Language Modelscited by 305
- 2024Diffusion for World Modeling: Visual Details Matter in Ataricited by 300
- 2024Diffusion Models Are Real-Time Game Enginescited by 296
- 2020Scan2Cap: Context-aware Dense Captioning in RGB-D Scanscited by 218
- 2023LL3DA: Visual Interactive Instruction Tuning for Omni-3D Understanding, Reasoning, and Planningcited by 210
- 20233D-VisTA: Pre-trained Transformer for 3D Vision and Text Alignmentcited by 204
- 2024PIVOT: Iterative Visual Prompting Elicits Actionable Knowledge for VLMscited by 190
- 2024LLaVA-3D: A Simple yet Effective Pathway to Empowering LMMs with 3D-awarenesscited by 183
- 2024Scene-LLM: Extending Language Model for 3D Visual Understanding and Reasoningcited by 175
- 2025WorldMem: Long-term Consistent World Simulation with Memorycited by 168
- 2025VLM-3R: Vision-Language Models Augmented with Instruction-Aligned 3D Reconstructioncited by 166
- 2023Physically Grounded Vision-Language Models for Robotic Manipulationcited by 165
- 2025SpaceR: Reinforcing MLLMs in Video Spatial Reasoningcited by 159
- 2024SpatialBot: Precise Spatial Understanding with Vision Language Modelscited by 152
- 2023Multi3DRefer: Grounding Text Description to Multiple 3D Objectscited by 146
- 20243DSRBench: A Comprehensive 3D Spatial Reasoning Benchmarkcited by 139
- 2023Towards Learning a Generalist Model for Embodied Navigationcited by 138
- 2024Video-3D LLM: Learning Position-Aware Video Representation for 3D Scene Understandingcited by 138
- 2025From Flatland to Space: Teaching Vision-Language Models to Perceive and Reason in 3Dcited by 138
- 2023Chat-3D: Data-efficiently Tuning Large Language Model for Universal Dialogue of 3D Scenescited by 129
- 2024GameGen-X: Interactive Open-world Game Video Generationcited by 128
- 2025RoboRefer: Towards Spatial Referring with Reasoning in Vision-Language Models for Roboticscited by 127
- 2025Learning from Videos for 3D World: Enhancing MLLMs with 3D Vision Geometry Priorscited by 120
- 2024Grounded 3D-LLM with Referent Tokenscited by 115
- 2024SAT: Dynamic Spatial Aptitude Training for Multimodal Language Modelscited by 114
- 2025GPT4Scene: Understand 3D Scenes from Videos with Vision-Language Modelscited by 109
- 2025MindCube: Spatial Mental Modeling from Limited Viewscited by 95
50 most cited of 71 All 71 (JSON)
Built on by
Citing papers per year: 2026: 3.
3 papers.
Similar, not cited
Close in wording, no citation either way
- 2026SpatialImaginer: Towards Adaptive Visual Imagination for Spatial Reasoning
- 2026World2VLM: Distilling World Model Imagination into VLMs for Dynamic Spatial Reasoning
- 2025Imagine in Space: Exploring the Frontier of Spatial Intelligence and Reasoning Efficiency in Vision Language Models
- 2026Imaginative Perception Tokens Enhance Spatial Reasoning in Multimodal Language Models
- 2026SPARGen: Unifying Spatial Perception and Reasoning through Native Multimodal Generation
- 2026Thinking with Imagination: Agentic Visual Spatial Reasoning with World Simulators
- 2024SpatialRGPT: Grounded Spatial Reasoning in Vision Language Models
- 2025Thinking with Camera: A Unified Multimodal Model for Camera-Centric Understanding and Generation