mlsanityData to 7 Oct 2026

Revisiting the Othello World Model Hypothesis

arXiv 2503.04421 ↗First posted 6 Mar 2025cs.CLcited by 11

On the map: World models → World models for language agents

Abstract (arXiv)

Li et al. (2023) used the Othello board game as a test case for the ability of GPT-2 to induce world models, and were followed up by Nanda et al. (2023b). We briefly discuss the original experiments, expanding them to include more language models with more comprehensive probing. Specifically, we analyze sequences of Othello board states and train the model to predict the next move based on previous moves. We evaluate seven language models (GPT-2, T5, Bart, Flan-T5, Mistral, LLaMA-2, and Qwen2.5) on the Othello task and conclude that these models not only learn to play Othello, but also induce the Othello board layout. We find that all models achieve up to 99% accuracy in unsupervised grounding and exhibit high similarity in the board features they learned. This provides considerably stronger evidence for the Othello World Model Hypothesis than previous works.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Placed in World models for language agents, one of 11 directions in World models. Builds on 22 of its 34 references that are in the map. First posted on arXiv 6 Mar 2025.

Builds on

22 of 34 references.

Closest ancestors · 5

Same direction, newest first

  1. 2024Emergent World Models and Latent Variable Estimation in Chess-Playing Language Models21 Mar
  2. 2023Linear Latent World Models in Simple Transformers: A Case Study on Othello-GPT11 Oct
  3. 2023Emergent Linear Representations in World Models of Self-Supervised Sequence Models2 Sep
  4. 2023Reasoning with Language Model is Planning with World Model24 May
  5. 2023Language Models Meet World Models: Embodied Experiences Enhance Language Models18 May

Other references · 17

  1. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  4. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,462
  5. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  6. 2024Qwen2.5-Coder Technical Reportcited by 1,151
  7. 2017Word Translation Without Parallel Datacited by 480
  8. 2022Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Taskcited by 417
  9. 2024The Platonic Representation Hypothesiscited by 362
  10. 2017Unsupervised Machine Translation Using Monolingual Corpora Onlycited by 272
  11. 2021Implicit Representations of Meaning in Neural Language Modelscited by 179
  12. 2021Can Language Models Encode Perceptual Structure Without Grounding? A Case Study in Colorcited by 118
5 more
  1. 2024Can Language Models Serve as Text-Based World Simulators?cited by 50
  2. 2024mOthello: When Do Cross-Lingual Representation Alignment and Cross-Lingual Transfer Emerge in Multilingual Models?cited by 20
  3. 2023Structured World Representations in Maze-Solving Transformerscited by 16
  4. 2023Emergence of Abstract State Representations in Embodied Sequence Modelingcited by 9
  5. 2017Learning to Play Othello with Deep Neural Networkscited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 9.

11 papers.

  1. 2025Mechanistic Indicators of Understanding in Large Language Modelscited by 6
  2. 2026MetaOthello: A Controlled Study of Multiple World Models in Transformerscited by 3
  3. 2026Tool-Call Dependency Structure is Linearly Decodable in LLM Agent Residual Streamscited by 1
  4. 2025What if Othello-Playing Language Models Could See?cited by 0
  5. 2026Reasoning Capabilities of Large Language Models. Lessons Learned from General Game Playingcited by 0
  6. 2026Understanding the Emergence of Seemingly Useless Features in Next-Token Predictorscited by 0
  7. 2026When 2D Tasks Meet 1D Serialization: On Serialization Friction in Structured Taskscited by 0
  8. 2026A Close Look At World Model Recovery In Supervised Fine-Tuned LLM Plannerscited by 0
  9. 2026Autoregressive Mosaics: Probing 2D Spatial Reasoning in Text-Only Language Modelscited by 0
  10. 2026Few-Shot Demonstrations Elicit the Use of In-Context World Representations in LLMscited by 0
  11. 2026Small transformers track Bayesian evidence for latent common causes via a context-invariant mechanismcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025How GPT learns layer by layer
  2. 2025Do You Get the Hint? Benchmarking LLMs on the Board Game Concept
  3. 2024Evaluating the World Model Implicit in a Generative Model
  4. 2024Causal Language Modeling Can Elicit Search and Reasoning Capabilities on Logic Puzzles
  5. 2026The Token Games: Evaluating Language Model Reasoning with Puzzle Duels
  6. 2023Large Language Models on the Chessboard: A Study on ChatGPT's Formal Language Comprehension and Complex Reasoning Skills
  7. 2024Can Large Language Models Play Games? A Case Study of A Self-Play Approach
  8. 2025TextGames: Learning to Self-Play Text-Based Puzzle Games via Language Model Reasoning