# Learning Chess With Language Models and Transformers

arXiv: https://arxiv.org/abs/2209.11902 · HTML: https://mlsanity.com/paper/2209.11902/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 17 references that are in the map.
- First posted on arXiv 24 Sep 2022.

## Builds on (most cited first, 4 of 4)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Acquisition of Chess Knowledge in AlphaZero](https://mlsanity.com/paper/2111.09259/) (2021)
- [Chess AI: Competing Paradigms for Machine Intelligence](https://mlsanity.com/paper/2109.11602/) (2021)

## Built on by (most cited first, 6 of 6)

- [Amortized Planning with Large-Scale Transformers: A Case Study on Chess](https://mlsanity.com/paper/2402.04494/) (2024)
- [Explore the Reasoning Capability of LLMs in the Chess Testbed](https://mlsanity.com/paper/2411.06655/) (2024)
- [LLM CHESS: Benchmarking Reasoning and Instruction-Following in LLMs through Chess](https://mlsanity.com/paper/2512.01992/) (2025)
- [Representation Matters for Mastering Chess: Improved Feature Representation in AlphaZero Outperforms Switching to Transformers](https://mlsanity.com/paper/2304.14918/) (2023)
- [Aspect-based Sentiment Evaluation of Chess Moves (ASSESS): an NLP-based Method for Evaluating Chess Strategies from Textbooks](https://mlsanity.com/paper/2405.06499/) (2024)
- [Mixture of Masters: Sparse Chess Language Models with Player Routing](https://mlsanity.com/paper/2602.04447/) (2026)

## Abstract (arXiv)

Representing a board game and its positions by text-based notation enables the possibility of NLP applications. Language models, can help gain insight into a variety of interesting problems such as unsupervised learning rules of a game, detecting player behavior patterns, player attribution, and ultimately learning the game to beat state of the art. In this study, we applied BERT models, first to the simple Nim game to analyze its performance in the presence of noise in a setup of a few-shot learning architecture. We analyzed the model performance via three virtual players, namely Nim Guru, Random player, and Q-learner. In the second part, we applied the game learning language model to the chess game, and a large set of grandmaster games with exhaustive encyclopedia openings. Finally, we have shown that model practically learns the rules of the chess game and can survive games against Stockfish at a category-A rating level.