mlsanityData to 7 Oct 2026

From Text to Space: Mapping Abstract Spatial Models in LLMs during a Grid-World Navigation Task

arXiv 2502.16690 ↗First posted 23 Feb 2025cs.AIcited by 10

On the map: LLM agents and tool use → Multimodal and embodied agents

Abstract (arXiv)

Understanding how large language models (LLMs) represent and reason about spatial information is crucial for building robust agentic systems that can navigate real and simulated environments. In this work, we investigate the influence of different text-based spatial representations on LLM performance and internal activations in a grid-world navigation task. By evaluating models of various sizes on a task that requires navigating toward a goal, we examine how the format used to encode spatial information impacts decision-making. Our experiments reveal that cartesian representations of space consistently yield higher success rates and path efficiency, with performance scaling markedly with model size. Moreover, probing LLaMA-3.1-8B revealed subsets of internal units, primarily located in intermediate layers, that robustly correlate with spatial features, such as the position of the agent in the grid or action correctness, regardless of how that information is represented, and are also activated by unrelated spatial reasoning tasks. This work advances our understanding of how LLMs process spatial information and provides valuable insights for developing more interpretable and robust agentic AI systems.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 37 of its 54 references that are in the map. First posted on arXiv 23 Feb 2025.

Builds on

37 of 54 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents18 Jan

Other references · 36

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  4. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  5. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  6. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  7. 2022Emergent Abilities of Large Language Modelscited by 2,483
  8. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
  9. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,307
  10. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,130
  11. 2023Progress measures for grokking via mechanistic interpretabilitycited by 822
  12. 2023Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formattingcited by 778
24 more
  1. 2024A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applicationscited by 428
  2. 2024Mechanistic Interpretability for AI Safety -- A Reviewcited by 415
  3. 2023Finding Neurons in a Haystack: Case Studies with Sparse Probingcited by 328
  4. 2023Minigrid & Miniworld: Modular & Customizable Reinforcement Learning Environments for Goal-Oriented Taskscited by 320
  5. 2023Language Models Represent Space and Timecited by 306
  6. 2018TextWorld: A Learning Environment for Text-based Gamescited by 304
  7. 2023Ghost in the Minecraft: Generally Capable Agents for Open-World Environments via Large Language Models with Text-based Knowledge and Memorycited by 288
  8. 2020Experience Grounds Languagecited by 241
  9. 2021Implicit Representations of Meaning in Neural Language Modelscited by 179
  10. 2021Can Language Models Encode Perceptual Structure Without Grounding? A Case Study in Colorcited by 118
  11. 2024Is A Picture Worth A Thousand Words? Delving Into Spatial Reasoning for Vision Language Modelscited by 111
  12. 2024Evaluating the World Model Implicit in a Generative Modelcited by 107
  13. 2024WorldCoder, a Model-Based LLM Agent: Building World Models by Writing Code and Interacting with the Environmentcited by 99
  14. 2024The Butterfly Effect of Altering Prompts: How Small Changes and Jailbreaks Affect Large Language Model Performancecited by 97
  15. 2022Finding Skill Neurons in Pre-trained Transformer-based Language Modelscited by 65
  16. 2023Evaluating Spatial Understanding of Large Language Modelscited by 65
  17. 2023Can Large Language Models be Good Path Planners? A Benchmark and Investigation on Spatial-temporal Reasoningcited by 49
  18. 2024Learning to Localize Objects Improves Spatial Reasoning in Visual-LLMscited by 48
  19. 2023The language of prompting: What linguistic properties make a prompt successful?cited by 43
  20. 2024How Susceptible are LLMs to Influence in Prompts?cited by 32
  21. 2024Towards Understanding Multi-Task Learning (Generalization) of LLMs via Detecting and Exploring Task-Specific Neuronscited by 23
  22. 2022Discovering Salient Neurons in Deep NLP Modelscited by 21
  23. 2023Advances in Embodied Navigation Using Large Language Models: A Surveycited by 14
  24. 2024Large Language Models Understand Layoutcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 5; 2026: 5.

10 papers.

  1. 2025Assessing the Value of Visual Input: A Benchmark of Multimodal Large Language Models for Robotic Path Planningcited by 4
  2. 2026CRAFT: Grounded Multi-Agent Coordination Under Partial Informationcited by 3
  3. 2025Navigating Motion Agents in Dynamic and Cluttered Environments through LLM Reasoningcited by 2
  4. 2025Towards Physics-informed Spatial Intelligence with Human Priors: An Autonomous Driving Pilot Studycited by 2
  5. 2026Do LLMs Build Spatial World Models? Evidence from Grid-World Maze Taskscited by 2
  6. 2025Can LLMs Learn to Map the World from Local Descriptions?cited by 1
  7. 2025Using Text-Based Life Trajectories from Swedish Register Data to Predict Residential Mobility with Pretrained Transformerscited by 1
  8. 2026LLMs for Text-Based Exploration and Navigation Under Partial Observabilitycited by 1
  9. 2026Evaluating Few-Shot Temporal Reasoning of LLMs for Human Activity Prediction in Smart Environmentscited by 0
  10. 2026Do LLMs Build World Models From Text? A Multilingual Diagnostic of Spatial Reasoningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026From Human Cognition to Neural Activations: Probing the Computational Primitives of Spatial Reasoning in LLMs
  2. 2026Spatial Strategies, Not Actions: Vector-Quantized Geodesics as Tools for LLM-Driven Agents
  3. 2026The Sword, Shield, and Achilles' Heel: Characterizing the Linguistic Inductive Bias of Large Language Models for Spatial Reasoning in Navigation Planning
  4. 2025MazeEval: A Benchmark for Testing Sequential Decision-Making in Language Models
  5. 2025Cognitive Maps in Language Models: A Mechanistic Analysis of Spatial Planning
  6. 2025Stuck in the Matrix: Probing Spatial Reasoning in Large Language Models
  7. 2025LTD-Bench: Evaluating Large Language Models by Letting Them Draw
  8. 2026Grid Spatial Understanding: A Dataset for Textual Spatial Reasoning over Grids, Embodied Settings, and Coordinate Structures