# Code World Models for General Game Playing

arXiv: https://arxiv.org/abs/2510.04542 · HTML: https://mlsanity.com/paper/2510.04542/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Placed in World models for language agents, one of 11 directions in World models.
- Builds on 20 of its 37 references that are in the map.
- First posted on arXiv 6 Oct 2025.

## Placed in

- World models / [World models for language agents](https://mlsanity.com/wm/world-models-for-language-agents/)
- LLM reasoning / [Reasoning for agents, planning and code](https://mlsanity.com/lr/reasoning-for-agents-planning-and-code/)

## Builds on (most cited first, 20 of 20)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [OpenSpiel: A Framework for Reinforcement Learning in Games](https://mlsanity.com/paper/1908.09453/) (2019)
- [A Survey on Large Language Model-Based Game Agents](https://mlsanity.com/paper/2404.02039/) (2024)
- [WorldCoder, a Model-Based LLM Agent: Building World Models by Writing Code and Interacting with the Environment](https://mlsanity.com/paper/2402.12275/) (2024)
- [GameBench: Evaluating Strategic Reasoning Abilities of LLM Agents](https://mlsanity.com/paper/2406.06613/) (2024)
- [lmgame-Bench: How Good are LLMs at Playing Games?](https://mlsanity.com/paper/2505.15146/) (2025)
- [Mastering Board Games by External and Internal Planning with Language Models](https://mlsanity.com/paper/2412.12119/) (2024)
- [TextArena](https://mlsanity.com/paper/2504.11442/) (2025)
- [Measuring General Intelligence with Generated Games](https://mlsanity.com/paper/2505.07215/) (2025)
- [Assessing Adaptive World Models in Machines with Novel Games](https://mlsanity.com/paper/2507.12821/) (2025)
- [Game Theory Meets Large Language Models: A Systematic Survey with Taxonomy and New Frontiers](https://mlsanity.com/paper/2502.09053/) (2025)
- [Are Large Language Models Reliable AI Scientists? Assessing Reverse-Engineering of Black-Box Systems](https://mlsanity.com/paper/2505.17968/) (2025)
- [LLM-Guided Probabilistic Program Induction for POMDP Model Estimation](https://mlsanity.com/paper/2505.02216/) (2025)
- [Game Reasoning Arena: A Framework and Benchmark for Assessing Reasoning Capabilities of Large Language Models via Game Play](https://mlsanity.com/paper/2508.03368/) (2025)
- [Reinforcement Learning: An Overview](https://mlsanity.com/paper/2412.05265/) (2024)
- [Think in Games: Learning to Reason in Games via Reinforcement Learning with Large Language Models](https://mlsanity.com/paper/2508.21365/) (2025)
- [Autoformalization of Game Descriptions using Large Language Models](https://mlsanity.com/paper/2409.12300/) (2024)
- [Generative Agents for Multi-Agent Autoformalization of Interaction Scenarios](https://mlsanity.com/paper/2412.08805/) (2024)

## Built on by (most cited first, 26 of 26)

- [AutoHarness: improving LLM agents by automatically synthesizing a code harness](https://mlsanity.com/paper/2603.03329/) (2026)
- [Qwen-AgentWorld: Language World Models for General Agents](https://mlsanity.com/paper/2606.24597/) (2026)
- [Agentic World Modeling: Foundations, Capabilities, Laws, and Beyond](https://mlsanity.com/paper/2604.22748/) (2026)
- [Code World Model: Coding Agent as World Brain](https://mlsanity.com/paper/2608.25927/) (2026)
- [Code as Worlds: Agentic Discovery of Executable World Representations for Physical Reasoning](https://mlsanity.com/paper/2608.27549/) (2026)
- [Generative Visual Code Mobile World Models](https://mlsanity.com/paper/2602.01576/) (2026)
- [AI Gamestore: Scalable, Open-Ended Evaluation of Machine General Intelligence with Human Games](https://mlsanity.com/paper/2602.17594/) (2026)
- [PatchWorld: Gradient-Free Optimization of Executable World Models for Agent Environments](https://mlsanity.com/paper/2605.30880/) (2026)
- [Self-Improving World Modelling with Latent Actions](https://mlsanity.com/paper/2602.06130/) (2026)
- [Debugging code world models](https://mlsanity.com/paper/2602.07672/) (2026)
- [Distilling Game Code World Model Generation into Lightweight Large Language Models](https://mlsanity.com/paper/2605.24375/) (2026)
- [Towards Generalist Game Players: An Investigation of Foundation Models in the Game Multiverse](https://mlsanity.com/paper/2605.09965/) (2026)
- [AI for Games in the Foundation Model Era](https://mlsanity.com/paper/2609.16679/) (2026)
- [VisualPatchWorld: Code World Models as Latent Structured Representations for Planning](https://mlsanity.com/paper/2607.25236/) (2026)
- [Joint Learning of Hierarchical Neural Options and Abstract World Model](https://mlsanity.com/paper/2602.02799/) (2026)
- [Baba in Wonderland: Online Self-Supervised Dynamics Discovery for Executable World Models](https://mlsanity.com/paper/2605.16725/) (2026)
- [Abstraction Agent](https://mlsanity.com/paper/2609.04303/) (2026)
- [World-Time Compute with Verified Code World Models](https://mlsanity.com/paper/2609.09163/) (2026)
- [EMPIRIC: Experiment-Driven Learning of Residual World Models for Robot Planning](https://mlsanity.com/paper/2609.35047/) (2026)
- [Towards Improving Sequential Decision-Making in LLM Agents via Experience Memory](https://mlsanity.com/paper/2608.03420/) (2026)
- [Position: Profiling Game Worlds by Transition Complexity](https://mlsanity.com/paper/2608.18079/) (2026)
- [WMLLM: Self-Evolving Optimization Agents via Predict-Then-Act World Modeling](https://mlsanity.com/paper/2609.01608/) (2026)
- [COUNTERMEM: World-Model Verified Counter-Factual Memory for Language Agents](https://mlsanity.com/paper/2609.31874/) (2026)
- [Memento 3: Model-Based Recursive Self-Improvement through Reflective Rulebooks](https://mlsanity.com/paper/2610.11794/) (2026)
- [AgentGarten: Code Worlds for Evolving Agents](https://mlsanity.com/paper/2610.12374/) (2026)
- [Player-1: Learning to Play from Synthetic Experience](https://mlsanity.com/paper/2610.20584/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) reasoning abilities are increasingly being applied to classical board and card games, but the dominant approach -- involving prompting for direct move generation -- has significant drawbacks. It relies on the model's implicit fragile pattern-matching capabilities, leading to frequent illegal moves and strategically shallow play. Here we introduce an alternative approach: We use the LLM to translate natural language rules and game trajectories into a formal, executable world model represented as Python code. This generated model -- comprising functions for state transition, legal move enumeration, and termination checks -- serves as a verifiable simulation engine for high-performance planning algorithms like Monte Carlo tree search (MCTS). In addition, we prompt the LLM to generate heuristic value functions (to make MCTS more efficient), and inference functions (to estimate hidden states in imperfect information games). Our method offers three distinct advantages compared to directly using the LLM as a policy: (1) Verifiability: The generated CWM serves as a formal specification of the game's rules, allowing planners to algorithmically enumerate valid actions and avoid illegal moves, contingent on the correctness of the synthesized model; (2) Strategic Depth: We combine LLM semantic understanding with the deep search power of classical planners; and (3) Generalization: We direct the LLM to focus on the meta-task of data-to-code translation, enabling it to adapt to new games more easily. We evaluate our agent on 10 different games, of which 4 are novel and created for this paper. 5 of the games are fully observed (perfect information), and 5 are partially observed (imperfect information). We find that our method outperforms or matches Gemini 2.5 Pro in 9 out of the 10 considered games.