# Complete Chess Games Enable LLM Become A Chess Master

arXiv: https://arxiv.org/abs/2501.17186 · HTML: https://mlsanity.com/paper/2501.17186/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 26 references that are in the map.
- First posted on arXiv 26 Jan 2025.

## Builds on (most cited first, 16 of 16)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Deep reinforcement learning from human preferences](https://mlsanity.com/paper/1706.03741/) (2017)
- [Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling](https://mlsanity.com/paper/2304.01373/) (2023)
- [GLM: General Language Model Pretraining with Autoregressive Blank Infilling](https://mlsanity.com/paper/2103.10360/) (2021)
- [OpenAssistant Conversations -- Democratizing Large Language Model Alignment](https://mlsanity.com/paper/2304.07327/) (2023)
- [GPT-NeoX-20B: An Open-Source Autoregressive Language Model](https://mlsanity.com/paper/2204.06745/) (2022)
- [Universal Jailbreak Backdoors from Poisoned Human Feedback](https://mlsanity.com/paper/2311.14455/) (2023)
- [Acquisition of Chess Knowledge in AlphaZero](https://mlsanity.com/paper/2111.09259/) (2021)
- [ChessGPT: Bridging Policy Learning and Language Modeling](https://mlsanity.com/paper/2306.09200/) (2023)
- [Exploring Mathematical Extrapolation of Large Language Models with Synthetic Data](https://mlsanity.com/paper/2406.02100/) (2024)

## Built on by (most cited first, 16 of 16)

- [Xiangqi-R1: Enhancing Spatial Strategic Reasoning in LLMs for Chinese Chess via Reinforcement Learning](https://mlsanity.com/paper/2507.12215/) (2025)
- [OpaqueToolsBench: Learning Nuances of Tool Behavior Through Interaction](https://mlsanity.com/paper/2602.15197/) (2026)
- [Towards Generalist Game Players: An Investigation of Foundation Models in the Game Multiverse](https://mlsanity.com/paper/2605.09965/) (2026)
- [Baba is LLM: Reasoning in a Game with Dynamic Rules](https://mlsanity.com/paper/2506.19095/) (2025)
- [Out-of-distribution Tests Reveal Compositionality in Chess Transformers](https://mlsanity.com/paper/2510.20783/) (2025)
- [Mixing Expert Knowledge: Bring Human Thoughts Back To the Game of Go](https://mlsanity.com/paper/2601.16447/) (2026)
- [SKATE, a Scalable Tournament Eval: Weaker LLMs differentiate between stronger ones using verifiable challenges](https://mlsanity.com/paper/2508.06111/) (2025)
- [Emergent World Beliefs: Exploring Transformers in Stochastic Games](https://mlsanity.com/paper/2512.23722/) (2025)
- [Mixture of Masters: Sparse Chess Language Models with Player Routing](https://mlsanity.com/paper/2602.04447/) (2026)
- [VAM: Verbalized Action Masking for Controllable Exploration in RL Post-Training -- A Chess Case Study](https://mlsanity.com/paper/2602.16833/) (2026)
- [Causal Masking on Spatial Data: An Information-Theoretic Case for Learning Spatial Datasets with Unimodal Language Models](https://mlsanity.com/paper/2510.27009/) (2025)
- [Liquid Reasoning Transformers: A Sudoku-Based Prototype for Chess-Scale Algorithmic Tasks](https://mlsanity.com/paper/2512.12792/) (2025)
- [Tracking vs. Deciding: The Dual-Capability Bottleneck in Searchless Chess Transformers](https://mlsanity.com/paper/2603.29761/) (2026)
- [Generalization or Memorization? Brittleness Testing for Chess-Trained Language Models](https://mlsanity.com/paper/2605.17565/) (2026)
- [Communicating Chess Strategies in Natural Language](https://mlsanity.com/paper/2607.11486/) (2026)
- [Benchmarking Prompt Optimization of Large Language Models With Chess](https://mlsanity.com/paper/2610.00416/) (2026)

## Abstract (arXiv)

Large language models (LLM) have shown remarkable abilities in text generation, question answering, language translation, reasoning and many other tasks. It continues to advance rapidly and is becoming increasingly influential in various fields, from technology and business to education and entertainment. Despite LLM's success in multiple areas, its ability to play abstract games, such as chess, is underexplored. Chess-playing requires the language models to output legal and reasonable moves from textual inputs. Here, we propose the Large language model ChessLLM to play full chess games. We transform the game into a textual format with the best move represented in the Forsyth-Edwards Notation. We show that by simply supervised fine-tuning, our model has achieved a professional-level Elo rating of 1788 in matches against the standard Elo-rated Stockfish when permitted to sample 10 times. We further show that data quality is important. Long-round data supervision enjoys a 350 Elo rating improvement over short-round data.