SpatialTree: How Spatial Abilities Branch Out in MLLMs
Not on the map. Why?
Abstract (arXiv)
Cognitive science suggests that spatial ability develops progressively-from perception to reasoning and interaction. Yet in multimodal LLMs (MLLMs), this hierarchy remains poorly understood, as most studies focus on a narrow set of tasks. We introduce SpatialTree, a cognitive-science-inspired hierarchy that organizes spatial abilities into four levels: low-level perception (L1), mental mapping (L2), simulation (L3), and agentic competence (L4). Based on this taxonomy, we construct the first capability-centric hierarchical benchmark, thoroughly evaluating mainstream MLLMs across 27 sub-abilities. The evaluation results reveal a clear structure: L1 skills are largely orthogonal, whereas higher-level skills are strongly correlated, indicating increasing interdependency. Through targeted supervised fine-tuning, we uncover a surprising transfer dynamic-negative transfer within L1, but strong cross-level transfer from low- to high-level abilities with notable synergy. Finally, we explore how to improve the entire hierarchy. We find that naive RL that encourages extensive "thinking" is unreliable: it helps complex reasoning but hurts intuitive perception. We propose a simple auto-think strategy that suppresses unnecessary deliberation, enabling RL to consistently improve performance across all levels. By building SpatialTree, we provide a proof-of-concept framework for understanding and systematically scaling spatial abilities in MLLMs.
Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Builds on 50 of its 64 references that are in the map. First posted on arXiv 23 Dec 2025.
Builds on
50 of 64 references.
References · 50
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2023GPT-4 Technical Reportcited by 18,396
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
- 2025Qwen3 Technical Reportcited by 8,820
- 2023Visual Instruction Tuningcited by 7,869
- 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
- 2025Qwen2.5-VL Technical Reportcited by 5,587
- 2024GPT-4o System Cardcited by 4,588
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
- 2023Qwen Technical Reportcited by 3,344
- 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
- 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,376
38 more
- 2024Depth Anything: Unleashing the Power of Large-Scale Unlabeled Datacited by 1,360
- 2024DROID: A Large-Scale In-The-Wild Robot Manipulation Datasetcited by 852
- 2021Common Objects in 3D: Large-Scale Learning and Evaluation of Real-life 3D Category Reconstructioncited by 652
- 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spacescited by 611
- 2025UI-TARS: Pioneering Automated GUI Interaction with Native Agentscited by 610
- 2020Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understandingcited by 604
- 2024LLaVA-NeXT-Interleave: Tackling Multi-image, Video, and 3D in Large Multimodal Modelscited by 584
- 2024BLINK: Multimodal Large Language Models Can See but Not Perceivecited by 550
- 2020TartanAir: A Dataset to Push the Limits of Visual SLAMcited by 506
- 2021ARKitScenes: A Diverse Real-World Dataset For 3D Indoor Scene Understanding Using Mobile RGB-D Datacited by 454
- 2025Kimi-VL Technical Reportcited by 388
- 2025Seed1.5-VL Technical Reportcited by 373
- 2023PointLLM: Empowering Large Language Models to Understand Point Cloudscited by 333
- 2025EmbodiedBench: Comprehensive Benchmarking Multi-modal Large Language Models for Vision-Driven Embodied Agentscited by 221
- 2025EgoDex: Learning Dexterous Manipulation from Large-Scale Egocentric Videocited by 211
- 2024LLaVA-3D: A Simple yet Effective Pathway to Empowering LMMs with 3D-awarenesscited by 183
- 2025MMSI-Bench: A Benchmark for Multi-Image Spatial Intelligencecited by 183
- 2024SpatialTracker: Tracking Any 2D Pixels in 3D Spacecited by 181
- 2024MiraData: A Large-Scale Video Dataset with Long Durations and Structured Captionscited by 146
- 20243DSRBench: A Comprehensive 3D Spatial Reasoning Benchmarkcited by 139
- 2025OmniSpatial: Towards Comprehensive Spatial Reasoning Benchmark for Vision Language Modelscited by 114
- 2024Is A Picture Worth A Thousand Words? Delving Into Spatial Reasoning for Vision Language Modelscited by 113
- 2025MindCube: Spatial Mental Modeling from Limited Viewscited by 95
- 2025Visual Spatial Tuningcited by 89
- 2025SpatialTrackerV2: 3D Point Tracking Made Easycited by 87
- 2022Learning Affordance Grounding from Exocentric Imagescited by 82
- 2025Yume: An Interactive World Generation Modelcited by 81
- 2025SpatialLM: Training Large Language Models for Structured Indoor Modelingcited by 65
- 2025Towards Understanding Camera Motions in Any Videocited by 64
- 2025Spatial Reasoning with Vision-Language Models in Ego-Centric Multi-View Scenescited by 63
- 2024GeoCalib: Learning Single-image Calibration with Geometric Optimizationcited by 52
- 2025Multi-SpatialMLLM: Multi-Frame Spatial Understanding with Multi-Modal Large Language Modelscited by 52
- 2024Orient Anything: Learning Robust Object Orientation Estimation from Rendering 3D Modelscited by 45
- 2025Trace Anything: Representing Any Video in 4D via Trajectory Fieldscited by 33
- 2025SITE: towards Spatial Intelligence Thorough Evaluationcited by 29
- 2025Spatial Reasoning in Multimodal Large Language Models: A Survey of Tasks, Benchmarks and Methodscited by 23
- 2025IR3D-Bench: Evaluating Vision-Language Model Scene Understanding as Agentic Inverse Renderingcited by 19
- 2025InfinityHuman: Towards Long-Term Audio-Driven Humancited by 9
Built on by
Citing papers per year: 2026: 12.
12 papers.
- 2026SpatialClaw: Rethinking Action Interface for Agentic Spatial Reasoningcited by 17
- 2026TwinBrainVLA: Unleashing the Potential of Generalist VLMs for Embodied Tasks via Asymmetric Mixture-of-Transformerscited by 14
- 2026VideoZeroBench: Probing the Limits of Video MLLMs with Spatio-Temporal Evidence Verificationcited by 7
- 20263D-Mix for VLA: A Plug-and-Play Module for Integrating VGGT-based 3D Information into Vision-Language-Action Modelscited by 6
- 2026OVO-S-Bench: A Hierarchical Benchmark for Streaming Spatial Intelligence in Multimodal LLMscited by 6
- 2026MindEdit-Bench: Benchmarking Object-Level Counterfactual Spatial Reasoning in VLMs from In-the-Wild Photoscited by 4
- 2026Show, Don't Tell: Evaluating Spatial Cognition in Generative Pixels Rather Than LLM Textcited by 2
- 2026SpatialAct: Probing Spatial Reasoning-to-Action Capabilities of VLM Agents in 3D Scenescited by 1
- 2026Evaluation as Evolution: Transforming Adversarial Diffusion into Closed-Loop Curricula for Autonomous Vehiclescited by 0
- 2026Spatial-IQ: Deconstructing Spatial Intelligence via Hierarchical Capability Testscited by 0
- 2026DriveHierarchy: A Benchmark for Diagnosing VLM Driving Capabilities from Open-Loop Understanding to Closed-Loop Executioncited by 0
- 2026EvoSpatial: Experience-Driven Skill and Tool Evolution for Spatial Reasoning Agentscited by 0
- 2026EvoSpatial: Experience-Driven Skill and Tool Evolution for Spatial Reasoning Agents12 Oct 2026
- 2026DriveHierarchy: A Benchmark for Diagnosing VLM Driving Capabilities from Open-Loop Understanding to Closed-Loop Execution25 Sep 2026
- 2026Spatial-IQ: Deconstructing Spatial Intelligence via Hierarchical Capability Tests24 Jul 2026
- 2026Show, Don't Tell: Evaluating Spatial Cognition in Generative Pixels Rather Than LLM Text23 Jul 2026
- 2026MindEdit-Bench: Benchmarking Object-Level Counterfactual Spatial Reasoning in VLMs from In-the-Wild Photos1 Jul 2026
- 2026SpatialClaw: Rethinking Action Interface for Agentic Spatial Reasoning11 Jun 2026
- 2026OVO-S-Bench: A Hierarchical Benchmark for Streaming Spatial Intelligence in Multimodal LLMs2 Jun 2026
- 2026SpatialAct: Probing Spatial Reasoning-to-Action Capabilities of VLM Agents in 3D Scenes29 May 2026
- 2026Evaluation as Evolution: Transforming Adversarial Diffusion into Closed-Loop Curricula for Autonomous Vehicles8 Apr 2026
- 2026VideoZeroBench: Probing the Limits of Video MLLMs with Spatio-Temporal Evidence Verification2 Apr 2026
- 20263D-Mix for VLA: A Plug-and-Play Module for Integrating VGGT-based 3D Information into Vision-Language-Action Models25 Mar 2026
- 2026TwinBrainVLA: Unleashing the Potential of Generalist VLMs for Embodied Tasks via Asymmetric Mixture-of-Transformers20 Jan 2026
Similar, not cited
Close in wording, no citation either way
- 202511Plus-Bench: Demystifying Multimodal LLM Spatial Reasoning with Cognitive-Inspired Analysis
- 2025Defining and Evaluating Visual Language Models' Basic Spatial Abilities: A Perspective from Psychometrics
- 2025SpatialBench: Benchmarking Multimodal Large Language Models for Spatial Cognition
- 2026From Where Things Are to What They Are For: Benchmarking Spatial-Functional Intelligence in Multimodal LLMs
- 2026From Human Cognition to Neural Activations: Probing the Computational Primitives of Spatial Reasoning in LLMs
- 2026Chain-of-Thought Degrades Visual Spatial Reasoning Capabilities of Multimodal LLMs
- 2024Mind's Eye of LLMs: Visualization-of-Thought Elicits Spatial Reasoning in Large Language Models
- 2025Spatial4D-Bench: A Versatile 4D Spatial Intelligence Benchmark