Thinking with Blueprints: Assisting Vision-Language Models in Spatial Reasoning via Structured Object Representation
On the map: LLM reasoning → Spatial and embodied reasoning
Abstract (arXiv)
Spatial reasoning -- the ability to perceive and reason about relationships in space -- advances vision-language models (VLMs) from visual perception toward spatial semantic understanding. Existing approaches either revisit local image patches, improving fine-grained perception but weakening global spatial awareness, or mark isolated coordinates, which capture object locations but overlook their overall organization. In this work, we integrate the cognitive concept of an object-centric blueprint into VLMs to enhance spatial reasoning. Given an image and a question, the model first constructs a JSON-style blueprint that records the positions, sizes, and attributes of relevant objects, and then reasons over this structured representation to produce the final answer. To achieve this, we introduce three key techniques: (1) blueprint-embedded reasoning traces for supervised fine-tuning to elicit basic reasoning skills; (2) blueprint-aware rewards in reinforcement learning to encourage the blueprint to include an appropriate number of objects and to align final answers with this causal reasoning; and (3) anti-shortcut data augmentation that applies targeted perturbations to images and questions, discouraging reliance on superficial visual or linguistic cues. Experiments show that our method consistently outperforms existing VLMs and specialized spatial reasoning models.
Key facts. Cited by 21 papers in the map (data through 14 Oct 2026). Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning. Builds on 26 of its 61 references that are in the map. First posted on arXiv 5 Jan 2026.
Builds on
26 of 61 references.
Closest ancestors · 5
Same direction, newest first
- 2025Reinforcing Spatial Reasoning in Vision-Language Models with Interwoven Thinking and Visual Drawing11 Jun
- 2025ST-VLM: Kinematic Instruction Tuning for Spatio-Temporal Reasoning in Vision-Language Models25 Mar
- 2025Beyond Semantics: Rediscovering Spatial Awareness in Vision-Language Models21 Mar
- 2025Why Is Spatial Reasoning Hard for VLMs? An Attention Mechanism Perspective on Focus Areas3 Mar
- 2023What's "up" with vision-language models? Investigating their struggle with spatial reasoning30 Oct
Other references · 21
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
- 2024GPT-4o System Cardcited by 4,588
- 2025FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Spacecited by 1,006
- 2023Shikra: Unleashing Multimodal LLM's Referential Dialogue Magiccited by 906
- 2024BLINK: Multimodal Large Language Models Can See but Not Perceivecited by 550
- 2025Reasoning Models Don't Always Say What They Thinkcited by 415
- 2025Pixel Reasoner: Incentivizing Pixel-Space Reasoning with Curiosity-Driven Reinforcement Learningcited by 278
- 2025Spatial-MLLM: Boosting MLLM Capabilities in Visual-based Spatial Intelligencecited by 217
- 2025Machine Mental Imagery: Empower Multimodal Reasoning with Latent Visual Tokenscited by 124
- 2025Latent Visual Reasoningcited by 104
- 2025Visionary-R1: Mitigating Shortcuts in Visual Reasoning with Reinforcement Learningcited by 87
- 2025Grounded Reinforcement Learning for Visual Reasoningcited by 83
9 more
- 2025Q-Frame: Query-aware Frame Selection and Multi-Resolution Adaptation for Video-LLMscited by 80
- 2023Visual Chain of Thought: Bridging Logical Gaps with Multimodal Infillingscited by 69
- 2025V-STaR: Benchmarking Video-LLMs on Video Spatio-Temporal Reasoningcited by 68
- 2025Grounded Chain-of-Thought for Multimodal Large Language Modelscited by 59
- 2025Latent Sketchpad: Sketching Visual Thoughts to Elicit Multimodal Reasoning in MLLMscited by 43
- 2025VLA-R1: Enhancing Reasoning in Vision-Language-Action Modelscited by 38
- 20253D Aware Region Prompted Vision Language Modelcited by 33
- 2024DocLayLLM: An Efficient Multi-modal Extension of Large Language Models for Text-rich Document Understandingcited by 23
- 2025DeFacto: Counterfactual Thinking with Images for Enforcing Evidence-Grounded and Faithful Reasoningcited by 10
Built on by
Citing papers per year: 2026: 21.
21 papers.
Sort:
By area:
- 2026Agentic World Modeling: Foundations, Capabilities, Laws, and Beyondcited by 14
- 2026GeoEyes: On-Demand Visual Focusing for Evidence-Grounded Understanding of Ultra-High-Resolution Remote Sensing Imagerycited by 9
- 2026SPPO: Sequence-Level PPO for Long-Horizon Reasoning Taskscited by 9
- 2026From Narrow to Panoramic Vision: Attention-Guided Cold-Start Reshapes Multimodal Reasoningcited by 8
- 2026Bad Seeing or Bad Thinking? Rewarding Perception for Multimodal Reasoningcited by 7
- 2026When Models Learn to Ask Why: Adaptive Causal Reasoning for Trustworthy Medical Vision-Language Modelscited by 6
- 2026Android Coach: Improve Online Agentic Training Efficiency with Single State Multiple Actionscited by 6
- 2026SlowBA: An efficiency backdoor attack towards VLM-based GUI agentscited by 3
- 2026Visual Enhanced Depth Scaling for Multimodal Latent Reasoningcited by 3
- 2026Dual Distribution Estimation for Zero-shot Noisy Test-Time Adaptation with VLMscited by 2
- 2026Shaping Schema via Language Representation as the Next Frontier for LLM Intelligence Expandingcited by 1
- 2026Ouroboros-Spatial: Closing the Data-Model Loop for Spatial Reasoningcited by 1
9 more
- 2026Deferred Exposure of Future Trajectories for Verifiable Reasoning in Autonomous Driving VLMscited by 1
- 2026Constraint-First Reasoning: A Training-Free Protocol for Exploiting Answer-Space Constraints in Mathematical Problem Solvingcited by 0
- 2026SCOUT: Unlocking Enhanced Spatial Reasoning via Structured Chain-of-Thought and Multi-Objective Process Rewardcited by 0
- 2026Aphanta: Diagnosing Task-Aligned Image-Edited Intermediates for Multimodal Reasoningcited by 0
- 2026LeRF: Learning Reference Coordinate Frames for Perspective Taking Reasoningcited by 0
- 2026S3: Spectral Null-Space Swap Makes Reasoning Models Efficientcited by 0
- 2026SpatialCORE: Confidence-Aware Grounded Spatial Reasoning in Large Vision--Language Modelscited by 0
- 2026Soft Spatial Reasoningcited by 0
- 2026From Reasoning Failures to Composable Video Spatial Intelligencecited by 0
- 2026From Reasoning Failures to Composable Video Spatial Intelligence1 Oct 2026
- 2026Soft Spatial Reasoning30 Sep 2026
- 2026SpatialCORE: Confidence-Aware Grounded Spatial Reasoning in Large Vision--Language Models30 Sep 2026
- 2026S3: Spectral Null-Space Swap Makes Reasoning Models Efficient29 Sep 2026
- 2026LeRF: Learning Reference Coordinate Frames for Perspective Taking Reasoning28 Sep 2026
- 2026Aphanta: Diagnosing Task-Aligned Image-Edited Intermediates for Multimodal Reasoning27 Aug 2026
- 2026SCOUT: Unlocking Enhanced Spatial Reasoning via Structured Chain-of-Thought and Multi-Objective Process Reward12 Aug 2026
- 2026Constraint-First Reasoning: A Training-Free Protocol for Exploiting Answer-Space Constraints in Mathematical Problem Solving5 Aug 2026
- 2026Deferred Exposure of Future Trajectories for Verifiable Reasoning in Autonomous Driving VLMs3 Aug 2026
- 2026Dual Distribution Estimation for Zero-shot Noisy Test-Time Adaptation with VLMs24 Jun 2026
- 2026Ouroboros-Spatial: Closing the Data-Model Loop for Spatial Reasoning10 Jun 2026
- 2026Bad Seeing or Bad Thinking? Rewarding Perception for Multimodal Reasoning13 May 2026
9 more
- 2026Shaping Schema via Language Representation as the Next Frontier for LLM Intelligence Expanding10 May 2026
- 2026Agentic World Modeling: Foundations, Capabilities, Laws, and Beyond24 Apr 2026
- 2026Visual Enhanced Depth Scaling for Multimodal Latent Reasoning12 Apr 2026
- 2026SPPO: Sequence-Level PPO for Long-Horizon Reasoning Tasks10 Apr 2026
- 2026Android Coach: Improve Online Agentic Training Efficiency with Single State Multiple Actions8 Apr 2026
- 2026When Models Learn to Ask Why: Adaptive Causal Reasoning for Trustworthy Medical Vision-Language Models24 Mar 2026
- 2026SlowBA: An efficiency backdoor attack towards VLM-based GUI agents9 Mar 2026
- 2026From Narrow to Panoramic Vision: Attention-Guided Cold-Start Reshapes Multimodal Reasoning4 Mar 2026
- 2026GeoEyes: On-Demand Visual Focusing for Evidence-Grounded Understanding of Ultra-High-Resolution Remote Sensing Imagery15 Feb 2026
- 2026SPPO: Sequence-Level PPO for Long-Horizon Reasoning Taskscited by 9
- 2026From Narrow to Panoramic Vision: Attention-Guided Cold-Start Reshapes Multimodal Reasoningcited by 8
- 2026Bad Seeing or Bad Thinking? Rewarding Perception for Multimodal Reasoningcited by 7
- 2026When Models Learn to Ask Why: Adaptive Causal Reasoning for Trustworthy Medical Vision-Language Modelscited by 6
- 2026Visual Enhanced Depth Scaling for Multimodal Latent Reasoningcited by 3
- 2026Ouroboros-Spatial: Closing the Data-Model Loop for Spatial Reasoningcited by 1
- 2026Deferred Exposure of Future Trajectories for Verifiable Reasoning in Autonomous Driving VLMscited by 1
- 2026Constraint-First Reasoning: A Training-Free Protocol for Exploiting Answer-Space Constraints in Mathematical Problem Solvingcited by 0
- 2026SCOUT: Unlocking Enhanced Spatial Reasoning via Structured Chain-of-Thought and Multi-Objective Process Rewardcited by 0
- 2026Aphanta: Diagnosing Task-Aligned Image-Edited Intermediates for Multimodal Reasoningcited by 0
- 2026LeRF: Learning Reference Coordinate Frames for Perspective Taking Reasoningcited by 0
- 2026S3: Spectral Null-Space Swap Makes Reasoning Models Efficientcited by 0
12 most cited of 15 All 15 (JSON)
- 2026From Reasoning Failures to Composable Video Spatial Intelligence1 Oct 2026
- 2026Soft Spatial Reasoning30 Sep 2026
- 2026SpatialCORE: Confidence-Aware Grounded Spatial Reasoning in Large Vision--Language Models30 Sep 2026
- 2026S3: Spectral Null-Space Swap Makes Reasoning Models Efficient29 Sep 2026
- 2026LeRF: Learning Reference Coordinate Frames for Perspective Taking Reasoning28 Sep 2026
- 2026Aphanta: Diagnosing Task-Aligned Image-Edited Intermediates for Multimodal Reasoning27 Aug 2026
- 2026SCOUT: Unlocking Enhanced Spatial Reasoning via Structured Chain-of-Thought and Multi-Objective Process Reward12 Aug 2026
- 2026Constraint-First Reasoning: A Training-Free Protocol for Exploiting Answer-Space Constraints in Mathematical Problem Solving5 Aug 2026
- 2026Deferred Exposure of Future Trajectories for Verifiable Reasoning in Autonomous Driving VLMs3 Aug 2026
- 2026Ouroboros-Spatial: Closing the Data-Model Loop for Spatial Reasoning10 Jun 2026
- 2026Bad Seeing or Bad Thinking? Rewarding Perception for Multimodal Reasoning13 May 2026
- 2026Visual Enhanced Depth Scaling for Multimodal Latent Reasoning12 Apr 2026
12 newest of 15 All 15 (JSON)
Similar, not cited
Close in wording, no citation either way
- 2025Mind the Gap: Benchmarking Spatial Reasoning in Vision-Language Models
- 2024Reasoning Paths with Reference Objects Elicit Quantitative Spatial Reasoning in Large Vision-Language Models
- 2026WM-VLM: Probing Internal World Models for Interleaved Visual-Textual Reasoning
- 2026SpatiaLab: Can Vision-Language Models Perform Spatial Reasoning in the Wild?
- 2025Imagine in Space: Exploring the Frontier of Spatial Intelligence and Reasoning Efficiency in Vision Language Models
- 2025Enhancing Spatial Reasoning through Visual and Textual Thinking
- 2026When More Is Less: A Systematic Analysis of Spatial and Commonsense Information for Visual Spatial Reasoning
- 2024Is A Picture Worth A Thousand Words? Delving Into Spatial Reasoning for Vision Language Models