# Do LLMs Build World Models From Text? A Multilingual Diagnostic of Spatial Reasoning

arXiv: https://arxiv.org/abs/2605.28277 · HTML: https://mlsanity.com/paper/2605.28277/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in World models for language agents, one of 11 directions in World models.
- Builds on 21 of its 55 references that are in the map.
- First posted on arXiv 27 May 2026.

## Placed in

- World models / [World models for language agents](https://mlsanity.com/wm/world-models-for-language-agents/)
- LLM reasoning / [Spatial and embodied reasoning](https://mlsanity.com/lr/spatial-and-embodied-reasoning/)

## Builds on (most cited first, 21 of 21)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [Mistral 7B](https://mlsanity.com/paper/2310.06825/) (2023)
- [Gemma 2: Improving Open Language Models at a Practical Size](https://mlsanity.com/paper/2408.00118/) (2024)
- [Show Your Work: Scratchpads for Intermediate Computation with Language Models](https://mlsanity.com/paper/2112.00114/) (2021)
- [SmolLM2: When Smol Goes Big -- Data-Centric Training of a Small Language Model](https://mlsanity.com/paper/2502.02737/) (2025)
- [Thoughts Are All Over the Place: On the Underthinking of o1-Like LLMs](https://mlsanity.com/paper/2501.18585/) (2025)
- [OptimalThinkingBench: Evaluating Over and Underthinking in LLMs](https://mlsanity.com/paper/2508.13141/) (2025)
- [Inference-Time Computations for LLM Reasoning and Planning: A Benchmark and Insights](https://mlsanity.com/paper/2502.12521/) (2025)
- [FloorplanQA: A Benchmark for Spatial Reasoning in LLMs using Structured Representations](https://mlsanity.com/paper/2507.07644/) (2025)
- [From Text to Space: Mapping Abstract Spatial Models in LLMs during a Grid-World Navigation Task](https://mlsanity.com/paper/2502.16690/) (2025)
- [FoREST: Frame of Reference Evaluation in Spatial Reasoning Tasks](https://mlsanity.com/paper/2502.17775/) (2025)
- [MazeEval: A Benchmark for Testing Sequential Decision-Making in Language Models](https://mlsanity.com/paper/2507.20395/) (2025)
- [Can LLMs See Without Pixels? Benchmarking Spatial Intelligence from Textual Descriptions](https://mlsanity.com/paper/2601.03590/) (2026)
- [SpaMEM: Benchmarking Dynamic Spatial Reasoning via Perception-Memory Integration in Embodied Environments](https://mlsanity.com/paper/2604.22409/) (2026)
- [LLM-BabyBench: Can Language Models Plan in Worlds They Can Simulate?](https://mlsanity.com/paper/2505.12135/) (2025)
- [SpatialText: A Pure-Text Cognitive Benchmark for Spatial Understanding in Large Language Models](https://mlsanity.com/paper/2603.03002/) (2026)
- [Do LLMs Build Spatial World Models? Evidence from Grid-World Maze Tasks](https://mlsanity.com/paper/2604.10690/) (2026)
- [Empirical Characterization of Inference-Time Elicited Probability Transformations in Large Language Models](https://mlsanity.com/paper/2603.19262/) (2026)
- [From Human Cognition to Neural Activations: Probing the Computational Primitives of Spatial Reasoning in LLMs](https://mlsanity.com/paper/2603.26323/) (2026)

## Abstract (arXiv)

Whether large language models (LLMs) construct internal spatial world models from pure-text descriptions remains contested, and whether such capabilities transfer across languages has not been systematically studied. We introduce MentalMap, a multilingual diagnostic benchmark with a six-level capability hierarchy (L0-L5) spanning atomic spatial facts to generative world-graph construction, together with four diagnostic axes probing frame of reference, reading-direction bias, reasoning-effort allocation, and hallucination. MentalMap is built from 100 ProcTHOR household scenes, covers eight typologically diverse languages plus a structured-text control, and contains 39 task families across 1,950 evaluation cells. Evaluating thirteen LLMs across scales and model families, we identify a universal L3 reasoning cliff: no model retains even half of its L0 performance on viewpoint reasoning once baseline atomic accuracy exceeds 40%. The cliff persists across languages, scales, and prompting strategies, while structured-output failures and reasoning patterns vary substantially across models. Human evaluation under the identical pure-text protocol reproduces the same failure pattern, suggesting that the bottleneck arises from text-only working memory constraints rather than being specific to current LLM architectures. Our findings reframe pure-text spatial reasoning as a multi-axis world-modeling problem and motivate multimodal and scratchpad-augmented reasoning as future directions.