# The Chess Transformer: Mastering Play using Generative Language Models

arXiv: https://arxiv.org/abs/2008.04057 · HTML: https://mlsanity.com/paper/2008.04057/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Aug 2020.

## Built on by (most cited first, 30 of 37)

- [Using cognitive psychology to understand GPT-3](https://mlsanity.com/paper/2206.14576/) (2022)
- [Large Language Models and Games: A Survey and Roadmap](https://mlsanity.com/paper/2402.18659/) (2024)
- [ChessGPT: Bridging Policy Learning and Language Modeling](https://mlsanity.com/paper/2306.09200/) (2023)
- [Chess as a Testbed for Language Model State Tracking](https://mlsanity.com/paper/2102.13249/) (2021)
- [Amortized Planning with Large-Scale Transformers: A Case Study on Chess](https://mlsanity.com/paper/2402.04494/) (2024)
- [Evidence of Learned Look-Ahead in a Chess-Playing Neural Network](https://mlsanity.com/paper/2406.00877/) (2024)
- [Maia-2: A Unified Model for Human-AI Alignment in Chess](https://mlsanity.com/paper/2409.20553/) (2024)
- [Easy2Hard-Bench: Standardized Difficulty Labels for Profiling LLM Performance and Generalization](https://mlsanity.com/paper/2409.18433/) (2024)
- [Transformer models: an introduction and catalog](https://mlsanity.com/paper/2302.07730/) (2023)
- [Transcendence: Generative Models Can Outperform The Experts That Train Them](https://mlsanity.com/paper/2406.11741/) (2024)
- [ChatGPT Participates in a Computer Science Exam](https://mlsanity.com/paper/2303.09461/) (2023)
- [Explore the Reasoning Capability of LLMs in the Chess Testbed](https://mlsanity.com/paper/2411.06655/) (2024)
- [Word2World: Generating Stories and Worlds through Large Language Models](https://mlsanity.com/paper/2405.06686/) (2024)
- [AbductionRules: Training Transformers to Explain Unexpected Inputs](https://mlsanity.com/paper/2203.12186/) (2022)
- [Assessing Logical Puzzle Solving in Large Language Models: Insights from a Minesweeper Case Study](https://mlsanity.com/paper/2311.07387/) (2023)
- [LLM CHESS: Benchmarking Reasoning and Instruction-Following in LLMs through Chess](https://mlsanity.com/paper/2512.01992/) (2025)
- [ING-VP: MLLMs cannot Play Easy Vision-based Games Yet](https://mlsanity.com/paper/2410.06555/) (2024)
- [Missed Connections: Lateral Thinking Puzzles for Large Language Models](https://mlsanity.com/paper/2404.11730/) (2024)
- [Puzzle Solving without Search or Human Knowledge: An Unnatural Language Approach](https://mlsanity.com/paper/2109.02797/) (2021)
- [Representation Matters for Mastering Chess: Improved Feature Representation in AlphaZero Outperforms Switching to Transformers](https://mlsanity.com/paper/2304.14918/) (2023)
- [GameTraversalBenchmark: Evaluating Planning Abilities Of Large Language Models Through Traversing 2D Game Maps](https://mlsanity.com/paper/2410.07765/) (2024)
- [Exploring Large Language Models for Word Games:Who is the Spy?](https://mlsanity.com/paper/2503.15235/) (2025)
- [Grounded Chess Reasoning in Language Models via Master Distillation](https://mlsanity.com/paper/2603.20510/) (2026)
- [Transformers for Object Detection in Large Point Clouds](https://mlsanity.com/paper/2209.15258/) (2022)
- [Correct Answers from Sound Reasoning: Verifiable Process Supervision for Language Models](https://mlsanity.com/paper/2605.12519/) (2026)
- [Connecting the Dots: Evaluating Abstract Reasoning Capabilities of LLMs Using the New York Times Connections Word Game](https://mlsanity.com/paper/2406.11012/) (2024)
- [Learning to Play Chess from Textbooks (LEAP): a Corpus for Evaluating Chess Moves based on Sentiment Analysis](https://mlsanity.com/paper/2310.20260/) (2023)
- [Out-of-distribution Tests Reveal Compositionality in Chess Transformers](https://mlsanity.com/paper/2510.20783/) (2025)
- [Predicting Human Chess Moves: An AI Assisted Analysis of Chess Games Using Skill-group Specific n-gram Language Models](https://mlsanity.com/paper/2512.01880/) (2025)
- [Aspect-based Sentiment Evaluation of Chess Moves (ASSESS): an NLP-based Method for Evaluating Chess Strategies from Textbooks](https://mlsanity.com/paper/2405.06499/) (2024)

## Abstract (arXiv)

This work demonstrates that natural language transformers can support more generic strategic modeling, particularly for text-archived games. In addition to learning natural language skills, the abstract transformer architecture can generate meaningful moves on a chessboard. With further fine-tuning, the transformer learns complex gameplay by training on 2.8 million chess games in Portable Game Notation. After 30,000 training steps, OpenAI's Generative Pre-trained Transformer (GPT-2) optimizes weights for 774 million parameters. This fine-tuned Chess Transformer generates plausible strategies and displays game formations identifiable as classic openings, such as English or the Slav Exchange. Finally, in live play, the novel model demonstrates a human-to-transformer interface that correctly filters illegal moves and provides a novel method to challenge the transformer's chess strategies. We anticipate future work will build on this transformer's promise, particularly in other strategy games where features can capture the underlying complex rule syntax from simple but expressive player annotations.