mlsanityData to 14 Oct 2026

Planning in a recurrent neural network that plays Sokoban

arXiv 2407.15421 ↗First posted 22 Jul 2024cs.LG cs.AIcited by 18

Not on the map. Why?

Abstract (arXiv)

Planning is essential for solving complex tasks, yet the internal mechanisms underlying planning in neural networks remain poorly understood. Building on prior work, we analyze a recurrent neural network (RNN) trained on Sokoban, a challenging puzzle requiring sequential, irreversible decisions. We find that the RNN has a causal plan representation which predicts its future actions about 50 steps in advance. The quality and length of the represented plan increases over the first few steps. We uncover a surprising behavior: the RNN "paces" in cycles to give itself extra computation at the start of a level, and show that this behavior is incentivized by training. Leveraging these insights, we extend the trained RNN to significantly larger, out-of-distribution Sokoban puzzles, demonstrating robust representations beyond the training regime. We open-source our model and code, and believe the neural network's interesting behavior makes it an excellent model organism to deepen our understanding of learned planning.

Key facts. Cited by 18 papers in the map (data through 14 Oct 2026). Builds on 40 of its 71 references that are in the map. First posted on arXiv 22 Jul 2024.

Builds on

40 of 71 references.

References · 40

  1. 2015Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcastingcited by 1,436
  2. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,420
  3. 2016Understanding intermediate layers using linear classifier probescited by 1,353
  4. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,302
  5. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,085
  6. 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 883
  7. 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 876
  8. 2023Progress measures for grokking via mechanistic interpretabilitycited by 839
  9. 2017Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithmcited by 734
  10. 2023Towards Automated Circuit Discovery for Mechanistic Interpretabilitycited by 724
  11. 2023Measuring Faithfulness in Chain-of-Thought Reasoningcited by 703
  12. 2024Scaling and evaluating sparse autoencoderscited by 553
28 more
  1. 2016Adaptive Computation Time for Recurrent Neural Networkscited by 506
  2. 2022Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Taskcited by 431
  3. 2021Causal Abstractions of Neural Networkscited by 422
  4. 2023Emergent Linear Representations in World Models of Self-Supervised Sequence Modelscited by 319
  5. 2023Language Models Represent Space and Timecited by 316
  6. 2019Risks from Learned Optimization in Advanced Machine Learning Systemscited by 258
  7. 2024Let's Think Dot by Dot: Hidden Computation in Transformer Language Modelscited by 191
  8. 2023Large Language Models can Strategically Deceive their Users when Put Under Pressurecited by 146
  9. 2023A Toy Model of Universality: Reverse Engineering How Networks Learn Group Operationscited by 144
  10. 2022Goal Misgeneralization: Why Correct Specifications Aren't Enough For Correct Goalscited by 126
  11. 2021Goal Misgeneralization in Deep Reinforcement Learningcited by 123
  12. 2021Acquisition of Chess Knowledge in AlphaZerocited by 123
  13. 2021Can You Learn an Algorithm? Generalizing from Easy to Hard Problems with Recurrent Networkscited by 117
  14. 2019An investigation of model-free planningcited by 66
  15. 2022EnvPool: A Highly Parallel Reinforcement Learning Environment Execution Enginecited by 62
  16. 2024Emergent World Models and Latent Variable Estimation in Chess-Playing Language Modelscited by 62
  17. 2019Environmental drivers of systematicity and generalization in a situated agentcited by 50
  18. 2024A Mechanistic Analysis of a Transformer Trained on a Symbolic Multi-Step Reasoning Taskcited by 49
  19. 2023Codebook Features: Sparse and Discrete Interpretability for Neural Networkscited by 48
  20. 2023Bridging the Human-AI Knowledge Gap: Concept Discovery and Transfer in AlphaZerocited by 42
  21. 2024Evidence of Learned Look-Ahead in a Chess-Playing Neural Networkcited by 34
  22. 2023Emergence of Maps in the Memories of Blind Navigation Agentscited by 33
  23. 2023Understanding and Controlling a Maze-Solving Policy Networkcited by 22
  24. 2023Towards Evaluating AI Systems for Moral Status Using Self-Reportscited by 22
  25. 2024Unlocking the Future: Exploring Look-Ahead Planning Mechanistic Interpretability in Large Language Modelscited by 22
  26. 2023Thinker: Learning to Plan and Actcited by 13
  27. 2021Datasets for Studying Generalization from Easy to Hard Examplescited by 12
  28. 2023Cleanba: A Reproducible and Efficient Distributed Reinforcement Learning Platformcited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 5; 2026: 13.

18 papers.

Sort:

By area:

  1. 2025To Steer or Not to Steer? Mechanistic Error Reduction with Abstention for Language Modelscited by 29
  2. 2026Propensity Inference: Environmental Contributors to LLM Behaviourcited by 7
  3. 2025Path Channels and Plan Extension Kernels: a Mechanistic Description of Planning in a Sokoban RNNcited by 4
  4. 2025Is Temporal Difference Learning the Gold Standard for Stitching in RL?cited by 4
  5. 2026From Perception to Action: An Interactive Benchmark for Vision Reasoningcited by 4
  6. 2025K-MSHC: Unmasking Minimally Sufficient Head Circuits in Large Language Models with Experiments on Syntactic Classification Taskscited by 2
  7. 2025ParaScopes: What do Language Models Activations Encode About Future Text?cited by 2
  8. 2026SokoBench: Evaluating Long-Horizon Planning and Reasoning in Large Language Modelscited by 2
  9. 2026A Behavioural and Representational Evaluation of Goal-Directedness in Language Model Agentscited by 1
  10. 2026What Is the Minimum Architecture for Prolepsis? Early Irrevocable Commitment Across Tasks in Small Transformerscited by 1
  11. 2026Latent Programming Horizons in Coding Agentscited by 1
  12. 2026Planning as Emergent Behavior in Reinforcement Learning with Relational Hidden Statescited by 1
6 more
  1. 2026Temporal Recurrence Favors Fewer Layerscited by 1
  2. 2026Structure and Scale in Simplicial Sequence Modellingcited by 0
  3. 2026The Shape of Addition: Geometric Structures of Arithmetic in Large Language Modelscited by 0
  4. 2026Reading the Room: Implicit Confusion Encoding in Recurrent World Model Statescited by 0
  5. 2026Understanding Decision-Making Mechanisms in Neural Routing Solverscited by 0
  6. 2026Looped Actor: Depth-Recurrent Reasoning Models for Reinforcement Learningcited by 0