# Can Large Language Models Play Games? A Case Study of A Self-Play Approach

arXiv: https://arxiv.org/abs/2403.05632 · HTML: https://mlsanity.com/paper/2403.05632/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Placed in Self-play RL for reasoning, one of 13 directions in Synthetic data and self-improvement for LLMs.
- Builds on 12 of its 36 references that are in the map.
- First posted on arXiv 8 Mar 2024.

## Placed in

- Synthetic data and self-improvement for LLMs / [Self-play RL for reasoning](https://mlsanity.com/synth/self-play-rl-for-reasoning/)

## Builds on (most cited first, 12 of 12)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Tree of Thoughts: Deliberate Problem Solving with Large Language Models](https://mlsanity.com/paper/2305.10601/) (2023)
- [A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions](https://mlsanity.com/paper/2311.05232/) (2023)
- [ALFWorld: Aligning Text and Embodied Environments for Interactive Learning](https://mlsanity.com/paper/2010.03768/) (2020)
- [Reasoning with Language Model is Planning with World Model](https://mlsanity.com/paper/2305.14992/) (2023)
- [Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Models](https://mlsanity.com/paper/2309.01219/) (2023)
- [Towards Reasoning in Large Language Models: A Survey](https://mlsanity.com/paper/2212.10403/) (2022)
- [The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A"](https://mlsanity.com/paper/2309.12288/) (2023)
- [Alphazero-like Tree-Search can Guide Large Language Model Decoding and Training](https://mlsanity.com/paper/2309.17179/) (2023)
- [Playing repeated games with Large Language Models](https://mlsanity.com/paper/2305.16867/) (2023)
- [Amortized Planning with Large-Scale Transformers: A Case Study on Chess](https://mlsanity.com/paper/2402.04494/) (2024)
- [Reason for Future, Act for Now: A Principled Framework for Autonomous LLM Agents with Provable Sample Efficiency](https://mlsanity.com/paper/2309.17382/) (2023)

## Built on by (most cited first, 13 of 13)

- [Large Language Model Agent: A Survey on Methodology, Applications and Challenges](https://mlsanity.com/paper/2503.21460/) (2025)
- [LLM as a Mastermind: A Survey of Strategic Reasoning with Large Language Models](https://mlsanity.com/paper/2404.01230/) (2024)
- [From AI for Science to Agentic Science: A Survey on Autonomous Scientific Discovery](https://mlsanity.com/paper/2508.14111/) (2025)
- [Game-theoretic LLM: Agent Workflow for Negotiation Games](https://mlsanity.com/paper/2411.05990/) (2024)
- [Explore the Reasoning Capability of LLMs in the Chess Testbed](https://mlsanity.com/paper/2411.06655/) (2024)
- [QSpec: Speculative Decoding with Complementary Quantization Schemes](https://mlsanity.com/paper/2410.11305/) (2024)
- [Game Theory Meets Large Language Models: A Systematic Survey with Taxonomy and New Frontiers](https://mlsanity.com/paper/2502.09053/) (2025)
- [LLM-Based Social Simulations Require a Boundary](https://mlsanity.com/paper/2506.19806/) (2025)
- [TextGames: Learning to Self-Play Text-Based Puzzle Games via Language Model Reasoning](https://mlsanity.com/paper/2502.18431/) (2025)
- [Beyond checkmate: exploring the creative chokepoints in AI text](https://mlsanity.com/paper/2501.19301/) (2025)
- [Shall We Play a Game? Language Models for Open-ended Wargames](https://mlsanity.com/paper/2509.17192/) (2025)
- [When LLM Meets Tree Search: A Systematic View of Inference as Search in Large Language Models](https://mlsanity.com/paper/2608.30395/) (2026)
- [Self-Play Search Distillation for Large Language Model Reasoning](https://mlsanity.com/paper/2609.30936/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) harness extensive data from the Internet, storing a broad spectrum of prior knowledge. While LLMs have proven beneficial as decision-making aids, their reliability is hampered by limitations in reasoning, hallucination phenomenon, and so on. On the other hand, Monte-Carlo Tree Search (MCTS) is a heuristic search algorithm that provides reliable decision-making solutions, achieved through recursive rollouts and self-play. However, the effectiveness of MCTS relies heavily on heuristic pruning and external value functions, particularly in complex decision scenarios. This work introduces an innovative approach that bolsters LLMs with MCTS self-play to efficiently resolve deterministic turn-based zero-sum games (DTZG), such as chess and go, without the need for additional training. Specifically, we utilize LLMs as both action pruners and proxies for value functions without the need for additional training. We theoretically prove that the suboptimality of the estimated value in our proposed method scales with tildemathcal OBigl(frac|tilde mathcal A|√N + εpruner + εcriticBigr), where (N) is the number of simulations, |tilde mathcal A| is the cardinality of the pruned action space by LLM, and εpruner and εcritic quantify the errors incurred by adopting LLMs as action space pruner and value function proxy, respectively. Our experiments in chess and go demonstrate the capability of our method to address challenges beyond the scope of MCTS and improve the performance of the directly application of LLMs.