mlsanityData to 7 Oct 2026

Grid Spatial Understanding: A Dataset for Textual Spatial Reasoning over Grids, Embodied Settings, and Coordinate Structures

arXiv 2603.17333 ↗First posted 18 Mar 2026cs.CLcited by 0

On the map: LLM reasoning → Spatial and embodied reasoning

Abstract (arXiv)

We introduce GSU, a text-only grid dataset to evaluate the spatial reasoning capabilities of LLMs over 3 core tasks: navigation, object localization, and structure composition. By forgoing visual inputs, isolating spatial reasoning from perception, we show that while most models grasp basic grid concepts, they struggle with frames of reference relative to an embodied agent and identifying 3D shapes from coordinate lists. We also find that exposure to a visual modality does not provide a generalizable understanding of 3D space that VLMs are able to utilize for these tasks. Finally, we show that while the very latest frontier models can solve the provided tasks (though harder variants may still stump them), fully fine-tuning a small LM or LORA fine-tuning a small LLM show potential to match frontier model performance, suggesting an avenue for specialized embodied agents.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning. Builds on 44 of its 51 references that are in the map. First posted on arXiv 18 Mar 2026.

Builds on

44 of 51 references.

Closest ancestors · 16

Same direction, newest first

  1. 2025Towards Geometry Problem Solving in the Large Model Era: A Survey3 Jun
  2. 2025Mind the Gap: Benchmarking Spatial Reasoning in Vision-Language Models25 Mar
  3. 2025iVISPAR -- An Interactive Visual-Spatial Reasoning Benchmark for VLMs5 Feb
  4. 2024Sparkle: Mastering Basic Spatial Capabilities in Vision Language Models Elicits Generalization to Spatial Reasoning21 Oct
  5. 2024Reasoning Paths with Reference Objects Elicit Quantitative Spatial Reasoning in Large Vision-Language Models15 Sep
  6. 2024Is A Picture Worth A Thousand Words? Delving Into Spatial Reasoning for Vision Language Models21 Jun
  7. 2024TopViewRS: Vision-Language Models as Top-View Spatial Reasoners4 Jun
  8. 2024SpatialRGPT: Grounded Spatial Reasoning in Vision Language Models3 Jun
  9. 2024Scene-LLM: Extending Language Model for 3D Visual Understanding and Reasoning18 Mar
  10. 2024SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilities22 Jan
  11. 2024Advancing Spatial Reasoning in Large Language Models: An In-Depth Evaluation and Enhancement Using the StepGame Benchmark8 Jan
  12. 2023Can Large Language Models be Good Path Planners? A Benchmark and Investigation on Spatial-temporal Reasoning5 Oct
4 more
  1. 20233D-LLM: Injecting the 3D World into Large Language Models24 Jul
  2. 2022UniGeo: Unifying Geometry Logical Reasoning via Reformulating Mathematical Expression6 Dec
  3. 2022Visual Spatial Reasoning30 Apr
  4. 2022StepGame: A New Benchmark for Robust Multi-Hop Spatial Reasoning in Texts18 Apr

Other references · 28

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  4. 2025Qwen3 Technical Reportcited by 8,195
  5. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  6. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  7. 2023Mistral 7Bcited by 3,065
  8. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  9. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,179
  10. 2024Qwen2 Technical Reportcited by 2,076
  11. 2025gpt-oss-120b & gpt-oss-20b Model Cardcited by 1,258
  12. 2023Gemini: A Family of Highly Capable Multimodal Modelscited by 810
16 more
  1. 2019ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Taskscited by 773
  2. 2024BLINK: Multimodal Large Language Models Can See but Not Perceivecited by 529
  3. 2024Large Language Models for Mathematical Reasoning: Progresses and Challengescited by 283
  4. 2019Vision-and-Dialog Navigationcited by 246
  5. 2025EmbodiedBench: Comprehensive Benchmarking Multi-modal Large Language Models for Vision-Driven Embodied Agentscited by 213
  6. 2023G-LLaVA: Solving Geometric Problem with Multi-Modal Large Language Modelcited by 208
  7. 2021TEACh: Task-driven Embodied Agents that Chatcited by 194
  8. 2024Pixtral 12Bcited by 174
  9. 2025Olmo 3cited by 95
  10. 2024Vision-and-Language Navigation Today and Tomorrow: A Survey in the Era of Foundation Modelscited by 79
  11. 2023Evaluating Cognitive Maps and Planning in Large Language Models with CogEvalcited by 77
  12. 2023Evaluating Spatial Understanding of Large Language Modelscited by 65
  13. 2025Embodied AI: From LLMs to World Modelscited by 23
  14. 2025Evaluation of LLMs for mathematical problem solvingcited by 7
  15. 2025BAP v2: An Enhanced Task Framework for Instruction Following in Minecraft Dialoguescited by 2
  16. 2025Reducing Latency in LLM-Based Natural Language Commands Processing for Robot Navigationcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Towards Physics-informed Spatial Intelligence with Human Priors: An Autonomous Driving Pilot Study
  2. 2025Stuck in the Matrix: Probing Spatial Reasoning in Large Language Models
  3. 2025REM: Evaluating LLM Embodied Spatial Reasoning through Multi-Frame Trajectories
  4. 2026Can LLMs See Without Pixels? Benchmarking Spatial Intelligence from Textual Descriptions
  5. 2026Chain of Spatial Thoughts: Modality-Agnostic Spatial Grounding for Vision Language Models
  6. 2026From Perception to Action: Spatial AI Agents and World Models
  7. 2025Spatial Reasoning in Multimodal Large Language Models: A Survey of Tasks, Benchmarks and Methods
  8. 2025SpatialCoT: Advancing Spatial Reasoning through Coordinate Alignment and Chain-of-Thought for Embodied Task Planning