# Linear Spatial World Models Emerge in Large Language Models

arXiv: https://arxiv.org/abs/2506.02996 · HTML: https://mlsanity.com/paper/2506.02996/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in World models for language agents, one of 11 directions in World models.
- Builds on 30 of its 49 references that are in the map.
- First posted on arXiv 3 Jun 2025.

## Placed in

- World models / [World models for language agents](https://mlsanity.com/wm/world-models-for-language-agents/)

## Builds on (most cited first, 30 of 30)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [Sparks of Artificial General Intelligence: Early experiments with GPT-4](https://mlsanity.com/paper/2303.12712/) (2023)
- [World Models](https://mlsanity.com/paper/1803.10122/) (2018)
- [Toy Models of Superposition](https://mlsanity.com/paper/2209.10652/) (2022)
- [Probing Classifiers: Promises, Shortcomings, and Advances](https://mlsanity.com/paper/2102.12452/) (2021)
- [The Linear Representation Hypothesis and the Geometry of Large Language Models](https://mlsanity.com/paper/2311.03658/) (2023)
- [Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Task](https://mlsanity.com/paper/2210.13382/) (2022)
- [Mechanistic Interpretability for AI Safety -- A Review](https://mlsanity.com/paper/2404.14082/) (2024)
- [The Platonic Representation Hypothesis](https://mlsanity.com/paper/2405.07987/) (2024)
- [Emergent Linear Representations in World Models of Self-Supervised Sequence Models](https://mlsanity.com/paper/2309.00941/) (2023)
- [Language Models Represent Space and Time](https://mlsanity.com/paper/2310.02207/) (2023)
- [The Alignment Problem from a Deep Learning Perspective](https://mlsanity.com/paper/2209.00626/) (2022)
- [Experience Grounds Language](https://mlsanity.com/paper/2004.10151/) (2020)
- [Foundational Challenges in Assuring Alignment and Safety of Large Language Models](https://mlsanity.com/paper/2404.09932/) (2024)
- [Probing the Probing Paradigm: Does Probing Accuracy Entail Task Relevance?](https://mlsanity.com/paper/2005.00719/) (2020)
- [Getting aligned on representational alignment](https://mlsanity.com/paper/2310.13018/) (2023)
- [Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networks](https://mlsanity.com/paper/2207.13243/) (2022)
- [Evaluating the World Model Implicit in a Generative Model](https://mlsanity.com/paper/2406.03689/) (2024)
- [Robust agents learn causal world models](https://mlsanity.com/paper/2402.10877/) (2024)
- [Chess as a Testbed for Language Model State Tracking](https://mlsanity.com/paper/2102.13249/) (2021)
- [Improving Activation Steering in Language Models with Mean-Centring](https://mlsanity.com/paper/2312.03813/) (2023)
- [Emergent World Models and Latent Variable Estimation in Chess-Playing Language Models](https://mlsanity.com/paper/2403.15498/) (2024)
- [From task structures to world models: What do LLMs know?](https://mlsanity.com/paper/2310.04276/) (2023)
- [Mathematical Models of Computation in Superposition](https://mlsanity.com/paper/2408.05451/) (2024)
- [Do Language Models Know the Way to Rome?](https://mlsanity.com/paper/2109.07971/) (2021)
- [Linear Latent World Models in Simple Transformers: A Case Study on Othello-GPT](https://mlsanity.com/paper/2310.07582/) (2023)
- [Structured World Representations in Maze-Solving Transformers](https://mlsanity.com/paper/2312.02566/) (2023)
- [World Models: The Safety Perspective](https://mlsanity.com/paper/2411.07690/) (2024)

## Built on by (most cited first, 6 of 6)

- [Self-Improving World Modelling with Latent Actions](https://mlsanity.com/paper/2602.06130/) (2026)
- [Cell-Based Representation of Relational Binding in Language Models](https://mlsanity.com/paper/2604.19052/) (2026)
- [Decodable Is Not Grounded: A Vision-Ablation Arbiter for VLM Spatial Reasoning](https://mlsanity.com/paper/2606.31257/) (2026)
- [Uncovering and Shaping the Latent Representation of 3D Scene Topology in Vision-Language Models](https://mlsanity.com/paper/2605.07148/) (2026)
- [World Modeling in Transformers](https://mlsanity.com/paper/2609.21748/) (2026)
- [From cacophony to hierarchy: a principled framework for assessing AI consciousness](https://mlsanity.com/paper/2609.35618/) (2026)

## Abstract (arXiv)

Large language models (LLMs) have demonstrated emergent abilities across diverse tasks, raising the question of whether they acquire internal world models. In this work, we investigate whether LLMs implicitly encode linear spatial world models, which we define as linear representations of physical space and object configurations. We introduce a formal framework for spatial world models and assess whether such structure emerges in contextual embeddings. Using a synthetic dataset of object positions, we train probes to decode object positions and evaluate geometric consistency of the underlying space. We further conduct causal interventions to test whether these spatial representations are functionally used by the model. Our results provide empirical evidence that LLMs encode linear spatial world models.