# Learning the Latent Rules of a Game from Data: A Chess Story

arXiv: https://arxiv.org/abs/2410.02426 · HTML: https://mlsanity.com/paper/2410.02426/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 36 references that are in the map.
- First posted on arXiv 3 Oct 2024.

## Builds on (most cited first, 6 of 6)

- [OPT: Open Pre-trained Transformer Language Models](https://mlsanity.com/paper/2205.01068/) (2022)
- [TinyStories: How Small Can Language Models Be and Still Speak Coherent English?](https://mlsanity.com/paper/2305.07759/) (2023)
- [LoRA Learns Less and Forgets Less](https://mlsanity.com/paper/2405.09673/) (2024)
- [Amortized Planning with Large-Scale Transformers: A Case Study on Chess](https://mlsanity.com/paper/2402.04494/) (2024)
- [Pretrained Generative Language Models as General Learning Frameworks for Sequence-Based Tasks](https://mlsanity.com/paper/2402.05616/) (2024)
- [Accurate Prediction of Ligand-Protein Interaction Affinities with Fine-Tuned Small Language Models](https://mlsanity.com/paper/2407.00111/) (2024)

## Built on by (most cited first, 1 of 1)

- [Explore the Reasoning Capability of LLMs in the Chess Testbed](https://mlsanity.com/paper/2411.06655/) (2024)

## Abstract (arXiv)

We demonstrate that small pretrained foundational generative language models with millions of parameters can learn the latent rules of a process from data associated with the process. Inspired by Stefan Zweig's novella "Schachnovelle," also known as "The Royal Game" in English, we show that 28M and 125M parameter pretrained foundational small language models (SLMs) can be instruction fine-tuned with 1,000-to-1,000,000 examples to learn the rules of chess, propose legal moves, and accurately solve chess problems. We also explore the impact of successive language model fine-tuning epochs on improved outcomes and demonstrate reductions in model hallucinations by increasing the number of instruction fine-tuning examples.