mlsanityData to 14 Oct 2026

Imaginative Perception Tokens Enhance Spatial Reasoning in Multimodal Language Models

arXiv 2606.03988 ↗First posted 2 Jun 2026cs.AIcited by 1

On the map: LLM reasoning → Spatial and embodied reasoning

also in 1 more
Abstract (arXiv)

Vision language models (VLMs) excel at many tasks but still struggle with spatial reasoning when critical information is not directly observable. Many such problems require imaginative perception: inferring what would be seen from an unseen viewpoint, tracing paths through occluded spaces, or integrating partial observations into a coherent spatial representation. We introduce Imaginative Perception Tokens (IPT), intermediate perceptual representations that externalize what a VLM would perceive under alternative spatial configurations while remaining consistent with the observed input. To study this capability, we formulate three tasks, Perspective Taking (PET), Path Tracing (PT), and Multiview Counting (MVC), and construct datasets of approximately 20K examples with ground truth imaginations, answers, and evaluation benchmarks. Using the unified VLM BAGEL as the backbone, IPT supervision consistently improves spatial reasoning and often outperforms textual chain of thought training, even without generating images at inference time. On MVC, IPT improves accuracy by 3.4% and achieves competitive performance with strong closed-source models on PT. We further find that combining IPT and label-only supervision yields additional gains, whereas textual chain of thought can substantially degrade performance, suggesting a modality mismatch when spatial computation is forced through language. Overall, IPT provides a principled supervision signal for reasoning about unobserved spatial structure, improving generalization while producing interpretable intermediate representations.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning. Builds on 32 of its 48 references that are in the map. First posted on arXiv 2 Jun 2026.

Builds on

32 of 48 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025OmniSpatial: Towards Comprehensive Spatial Reasoning Benchmark for Vision Language Models3 Jun
  2. 2025MMSI-Bench: A Benchmark for Multi-Image Spatial Intelligence29 May
  3. 2025ViewSpatial-Bench: Evaluating Multi-perspective Spatial Localization in Vision-Language Models27 May
  4. 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spaces18 Dec
  5. 2022Visual Spatial Reasoning30 Apr

Other references · 27

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  2. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,117
  3. 2025Qwen3-VL Technical Reportcited by 2,793
  4. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,376
  5. 2025SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Featurescited by 1,188
  6. 2025Qwen-Image Technical Reportcited by 1,044
  7. 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 927
  8. 2025Emerging Properties in Unified Multimodal Pretrainingcited by 897
  9. 2017AI2-THOR: An Interactive 3D Environment for Visual AIcited by 869
  10. 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 812
  11. 2023TIFA: Accurate and Interpretable Text-to-Image Faithfulness Evaluation with Question Answeringcited by 428
  12. 2022ProcTHOR: Large-Scale Embodied AI Using Procedural Generationcited by 380
15 more
  1. 2024Molmo and PixMo: Open Weights and Open Data for State-of-the-Art Vision-Language Modelscited by 329
  2. 2024Visual Sketchpad: Sketching as a Visual Chain of Thought for Multimodal Language Modelscited by 268
  3. 2025Show-o2: Improved Native Unified Multimodal Modelscited by 250
  4. 2025Imagine while Reasoning in Space: Multimodal Visualization-of-Thoughtcited by 206
  5. 2026Molmo2: Open Weights and Data for Vision-Language Models with Video Understanding and Groundingcited by 139
  6. 2025Machine Mental Imagery: Empower Multimodal Reasoning with Latent Visual Tokenscited by 124
  7. 2024SAT: Dynamic Spatial Aptitude Training for Multimodal Language Modelscited by 114
  8. 2025MindCube: Spatial Mental Modeling from Limited Viewscited by 95
  9. 2025Visual Spatial Tuningcited by 89
  10. 2024Perception Tokens Enhance Visual Reasoning in Multimodal Language Modelscited by 71
  11. 2025ThinkMorph: Emergent Properties in Multimodal Interleaved Chain-of-Thought Reasoningcited by 56
  12. 2025Benchmark Designers Should "Train on the Test Set" to Expose Exploitable Non-Visual Shortcutscited by 29
  13. 2025Mull-Tokens: Modality-Agnostic Latent Thinkingcited by 19
  14. 2025Seeing from Another Perspective: Evaluating Multi-View Understanding in MLLMscited by 16
  15. 2025SpatialViz-Bench: A Cognitively-Grounded Benchmark for Diagnosing Spatial Visualization in MLLMscited by 12

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2026SpatialImaginer: Towards Adaptive Visual Imagination for Spatial Reasoning
  2. 2025Imagine in Space: Exploring the Frontier of Spatial Intelligence and Reasoning Efficiency in Vision Language Models
  3. 2024Is A Picture Worth A Thousand Words? Delving Into Spatial Reasoning for Vision Language Models
  4. 2026World2VLM: Distilling World Model Imagination into VLMs for Dynamic Spatial Reasoning
  5. 2025Spatial Reasoning in Multimodal Large Language Models: A Survey of Tasks, Benchmarks and Methods
  6. 2026Chain of Spatial Thoughts: Modality-Agnostic Spatial Grounding for Vision Language Models
  7. 2026WM-VLM: Probing Internal World Models for Interleaved Visual-Textual Reasoning
  8. 2026When More Is Less: A Systematic Analysis of Spatial and Commonsense Information for Visual Spatial Reasoning