# Emergent World Models and Latent Variable Estimation in Chess-Playing Language Models

arXiv: https://arxiv.org/abs/2403.15498 · HTML: https://mlsanity.com/paper/2403.15498/

## Key facts

- Cited by 62 papers in the map (data through 14 Oct 2026).
- Placed in World models for language agents, one of 11 directions in World models.
- Builds on 12 of its 18 references that are in the map.
- First posted on arXiv 21 Mar 2024.

## Placed in

- World models / [World models for language agents](https://mlsanity.com/wm/world-models-for-language-agents/)

## Builds on (most cited first, 12 of 12)

- [Understanding intermediate layers using linear classifier probes](https://mlsanity.com/paper/1610.01644/) (2016)
- [Representation Engineering: A Top-Down Approach to AI Transparency](https://mlsanity.com/paper/2310.01405/) (2023)
- [Inference-Time Intervention: Eliciting Truthful Answers from a Language Model](https://mlsanity.com/paper/2306.03341/) (2023)
- [Editing Models with Task Arithmetic](https://mlsanity.com/paper/2212.04089/) (2022)
- [Steering Llama 2 via Contrastive Activation Addition](https://mlsanity.com/paper/2312.06681/) (2023)
- [Probing Classifiers: Promises, Shortcomings, and Advances](https://mlsanity.com/paper/2102.12452/) (2021)
- [Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Task](https://mlsanity.com/paper/2210.13382/) (2022)
- [Emergent Linear Representations in World Models of Self-Supervised Sequence Models](https://mlsanity.com/paper/2309.00941/) (2023)
- [Extracting Latent Steering Vectors from Pretrained Language Models](https://mlsanity.com/paper/2205.05124/) (2022)
- [Learning to Generate Reviews and Discovering Sentiment](https://mlsanity.com/paper/1704.01444/) (2017)
- [Acquisition of Chess Knowledge in AlphaZero](https://mlsanity.com/paper/2111.09259/) (2021)
- [Eight Things to Know about Large Language Models](https://mlsanity.com/paper/2304.00612/) (2023)

## Built on by (most cited first, 30 of 62)

- [Measuring Progress in Dictionary Learning for Language Model Interpretability with Board Game Models](https://mlsanity.com/paper/2408.00113/) (2024)
- [What Has a Foundation Model Found? Using Inductive Bias to Probe for World Models](https://mlsanity.com/paper/2507.06952/) (2025)
- [Evidence of Learned Look-Ahead in a Chess-Playing Neural Network](https://mlsanity.com/paper/2406.00877/) (2024)
- [(How) Do Language Models Track State?](https://mlsanity.com/paper/2503.02854/) (2025)
- [Controlling Large Language Model Agents with Entropic Activation Steering](https://mlsanity.com/paper/2406.00244/) (2024)
- [Mixture of Experts Made Intrinsically Interpretable](https://mlsanity.com/paper/2503.07639/) (2025)
- [Improving Reasoning Performance in Large Language Models via Representation Engineering](https://mlsanity.com/paper/2504.19483/) (2025)
- [Unlocking the Future: Exploring Look-Ahead Planning Mechanistic Interpretability in Large Language Models](https://mlsanity.com/paper/2406.16033/) (2024)
- [Transcendence: Generative Models Can Outperform The Experts That Train Them](https://mlsanity.com/paper/2406.11741/) (2024)
- [Planning in a recurrent neural network that plays Sokoban](https://mlsanity.com/paper/2407.15421/) (2024)
- [General agents contain world models](https://mlsanity.com/paper/2506.01622/) (2025)
- [Transformers Use Causal World Models in Maze-Solving Tasks](https://mlsanity.com/paper/2412.11867/) (2024)
- [Human-aligned Chess with a Bit of Search](https://mlsanity.com/paper/2410.03893/) (2024)
- [Implicit Search via Discrete Diffusion: A Study on Chess](https://mlsanity.com/paper/2502.19805/) (2025)
- [Revisiting the Othello World Model Hypothesis](https://mlsanity.com/paper/2503.04421/) (2025)
- [Levels of Analysis for Large Language Models](https://mlsanity.com/paper/2503.13401/) (2025)
- [Too Long, Didn't Model: Decomposing LLM Long-Context Understanding With Novels](https://mlsanity.com/paper/2505.14925/) (2025)
- [On Logical Extrapolation for Mazes with Recurrent and Implicit Networks](https://mlsanity.com/paper/2410.03020/) (2024)
- [Linear Spatial World Models Emerge in Large Language Models](https://mlsanity.com/paper/2506.02996/) (2025)
- [A Unified Definition of Hallucination: It's The World Model, Stupid!](https://mlsanity.com/paper/2512.21577/) (2025)
- [Chessformer: A Unified Architecture for Chess Modeling](https://mlsanity.com/paper/2605.19091/) (2026)
- [Do Language Models Track Entities Across State Changes?](https://mlsanity.com/paper/2605.30233/) (2026)
- [Path Channels and Plan Extension Kernels: a Mechanistic Description of Planning in a Sokoban RNN](https://mlsanity.com/paper/2506.10138/) (2025)
- [Better World Models Can Lead to Better Post-Training Performance](https://mlsanity.com/paper/2512.03400/) (2025)
- [MetaOthello: A Controlled Study of Multiple World Models in Transformers](https://mlsanity.com/paper/2602.23164/) (2026)
- [Enhancing Conversational Agents with Theory of Mind: Aligning Beliefs, Desires, and Intentions for Human-Like Interaction](https://mlsanity.com/paper/2502.14171/) (2025)
- [Beyond World Models: Rethinking Understanding in AI Models](https://mlsanity.com/paper/2511.12239/) (2025)
- [Baba is LLM: Reasoning in a Game with Dynamic Rules](https://mlsanity.com/paper/2506.19095/) (2025)
- [From Prediction to Understanding: Will AI Foundation Models Transform Brain Science?](https://mlsanity.com/paper/2509.17280/) (2025)
- [Hierarchical Experimentalist Agents](https://mlsanity.com/paper/2606.29315/) (2026)

## Abstract (arXiv)

Language models have shown unprecedented capabilities, sparking debate over the source of their performance. Is it merely the outcome of learning syntactic patterns and surface level statistics, or do they extract semantics and a world model from the text? Prior work by Li et al. investigated this by training a GPT model on synthetic, randomly generated Othello games and found that the model learned an internal representation of the board state. We extend this work into the more complex domain of chess, training on real games and investigating our model's internal representations using linear probes and contrastive activations. The model is given no a priori knowledge of the game and is solely trained on next character prediction, yet we find evidence of internal representations of board state. We validate these internal representations by using them to make interventions on the model's activations and edit its internal board state. Unlike Li et al's prior synthetic dataset approach, our analysis finds that the model also learns to estimate latent variables like player skill to better predict the next character. We derive a player skill vector and add it to the model, improving the model's win rate by up to 2.6 times.