mlsanityData to 14 Oct 2026

Thinking with Blueprints: Assisting Vision-Language Models in Spatial Reasoning via Structured Object Representation

arXiv 2601.01984 ↗First posted 5 Jan 2026cs.CVcited by 21

On the map: LLM reasoning → Spatial and embodied reasoning

also in 1 more
Abstract (arXiv)

Spatial reasoning -- the ability to perceive and reason about relationships in space -- advances vision-language models (VLMs) from visual perception toward spatial semantic understanding. Existing approaches either revisit local image patches, improving fine-grained perception but weakening global spatial awareness, or mark isolated coordinates, which capture object locations but overlook their overall organization. In this work, we integrate the cognitive concept of an object-centric blueprint into VLMs to enhance spatial reasoning. Given an image and a question, the model first constructs a JSON-style blueprint that records the positions, sizes, and attributes of relevant objects, and then reasons over this structured representation to produce the final answer. To achieve this, we introduce three key techniques: (1) blueprint-embedded reasoning traces for supervised fine-tuning to elicit basic reasoning skills; (2) blueprint-aware rewards in reinforcement learning to encourage the blueprint to include an appropriate number of objects and to align final answers with this causal reasoning; and (3) anti-shortcut data augmentation that applies targeted perturbations to images and questions, discouraging reliance on superficial visual or linguistic cues. Experiments show that our method consistently outperforms existing VLMs and specialized spatial reasoning models.

Key facts. Cited by 21 papers in the map (data through 14 Oct 2026). Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning. Builds on 26 of its 61 references that are in the map. First posted on arXiv 5 Jan 2026.

Builds on

26 of 61 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025Reinforcing Spatial Reasoning in Vision-Language Models with Interwoven Thinking and Visual Drawing11 Jun
  2. 2025ST-VLM: Kinematic Instruction Tuning for Spatio-Temporal Reasoning in Vision-Language Models25 Mar
  3. 2025Beyond Semantics: Rediscovering Spatial Awareness in Vision-Language Models21 Mar
  4. 2025Why Is Spatial Reasoning Hard for VLMs? An Attention Mechanism Perspective on Focus Areas3 Mar
  5. 2023What's "up" with vision-language models? Investigating their struggle with spatial reasoning30 Oct

Other references · 21

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  2. 2024GPT-4o System Cardcited by 4,588
  3. 2025FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Spacecited by 1,006
  4. 2023Shikra: Unleashing Multimodal LLM's Referential Dialogue Magiccited by 906
  5. 2024BLINK: Multimodal Large Language Models Can See but Not Perceivecited by 550
  6. 2025Reasoning Models Don't Always Say What They Thinkcited by 415
  7. 2025Pixel Reasoner: Incentivizing Pixel-Space Reasoning with Curiosity-Driven Reinforcement Learningcited by 278
  8. 2025Spatial-MLLM: Boosting MLLM Capabilities in Visual-based Spatial Intelligencecited by 217
  9. 2025Machine Mental Imagery: Empower Multimodal Reasoning with Latent Visual Tokenscited by 124
  10. 2025Latent Visual Reasoningcited by 104
  11. 2025Visionary-R1: Mitigating Shortcuts in Visual Reasoning with Reinforcement Learningcited by 87
  12. 2025Grounded Reinforcement Learning for Visual Reasoningcited by 83
9 more
  1. 2025Q-Frame: Query-aware Frame Selection and Multi-Resolution Adaptation for Video-LLMscited by 80
  2. 2023Visual Chain of Thought: Bridging Logical Gaps with Multimodal Infillingscited by 69
  3. 2025V-STaR: Benchmarking Video-LLMs on Video Spatio-Temporal Reasoningcited by 68
  4. 2025Grounded Chain-of-Thought for Multimodal Large Language Modelscited by 59
  5. 2025Latent Sketchpad: Sketching Visual Thoughts to Elicit Multimodal Reasoning in MLLMscited by 43
  6. 2025VLA-R1: Enhancing Reasoning in Vision-Language-Action Modelscited by 38
  7. 20253D Aware Region Prompted Vision Language Modelcited by 33
  8. 2024DocLayLLM: An Efficient Multi-modal Extension of Large Language Models for Text-rich Document Understandingcited by 23
  9. 2025DeFacto: Counterfactual Thinking with Images for Enforcing Evidence-Grounded and Faithful Reasoningcited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 21.

21 papers.

Sort:

By area:

  1. 2026Agentic World Modeling: Foundations, Capabilities, Laws, and Beyondcited by 14
  2. 2026GeoEyes: On-Demand Visual Focusing for Evidence-Grounded Understanding of Ultra-High-Resolution Remote Sensing Imagerycited by 9
  3. 2026SPPO: Sequence-Level PPO for Long-Horizon Reasoning Taskscited by 9
  4. 2026From Narrow to Panoramic Vision: Attention-Guided Cold-Start Reshapes Multimodal Reasoningcited by 8
  5. 2026Bad Seeing or Bad Thinking? Rewarding Perception for Multimodal Reasoningcited by 7
  6. 2026When Models Learn to Ask Why: Adaptive Causal Reasoning for Trustworthy Medical Vision-Language Modelscited by 6
  7. 2026Android Coach: Improve Online Agentic Training Efficiency with Single State Multiple Actionscited by 6
  8. 2026SlowBA: An efficiency backdoor attack towards VLM-based GUI agentscited by 3
  9. 2026Visual Enhanced Depth Scaling for Multimodal Latent Reasoningcited by 3
  10. 2026Dual Distribution Estimation for Zero-shot Noisy Test-Time Adaptation with VLMscited by 2
  11. 2026Shaping Schema via Language Representation as the Next Frontier for LLM Intelligence Expandingcited by 1
  12. 2026Ouroboros-Spatial: Closing the Data-Model Loop for Spatial Reasoningcited by 1
9 more
  1. 2026Deferred Exposure of Future Trajectories for Verifiable Reasoning in Autonomous Driving VLMscited by 1
  2. 2026Constraint-First Reasoning: A Training-Free Protocol for Exploiting Answer-Space Constraints in Mathematical Problem Solvingcited by 0
  3. 2026SCOUT: Unlocking Enhanced Spatial Reasoning via Structured Chain-of-Thought and Multi-Objective Process Rewardcited by 0
  4. 2026Aphanta: Diagnosing Task-Aligned Image-Edited Intermediates for Multimodal Reasoningcited by 0
  5. 2026LeRF: Learning Reference Coordinate Frames for Perspective Taking Reasoningcited by 0
  6. 2026S3: Spectral Null-Space Swap Makes Reasoning Models Efficientcited by 0
  7. 2026SpatialCORE: Confidence-Aware Grounded Spatial Reasoning in Large Vision--Language Modelscited by 0
  8. 2026Soft Spatial Reasoningcited by 0
  9. 2026From Reasoning Failures to Composable Video Spatial Intelligencecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Mind the Gap: Benchmarking Spatial Reasoning in Vision-Language Models
  2. 2024Reasoning Paths with Reference Objects Elicit Quantitative Spatial Reasoning in Large Vision-Language Models
  3. 2026WM-VLM: Probing Internal World Models for Interleaved Visual-Textual Reasoning
  4. 2026SpatiaLab: Can Vision-Language Models Perform Spatial Reasoning in the Wild?
  5. 2025Imagine in Space: Exploring the Frontier of Spatial Intelligence and Reasoning Efficiency in Vision Language Models
  6. 2025Enhancing Spatial Reasoning through Visual and Textual Thinking
  7. 2026When More Is Less: A Systematic Analysis of Spatial and Commonsense Information for Visual Spatial Reasoning
  8. 2024Is A Picture Worth A Thousand Words? Delving Into Spatial Reasoning for Vision Language Models