# Mixture of Masters: Sparse Chess Language Models with Player Routing

arXiv: https://arxiv.org/abs/2602.04447 · HTML: https://mlsanity.com/paper/2602.04447/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 46 of its 63 references that are in the map.
- First posted on arXiv 4 Feb 2026.

## Builds on (most cited first, 30 of 46)

- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity](https://mlsanity.com/paper/2101.03961/) (2021)
- [Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models](https://mlsanity.com/paper/2206.04615/) (2022)
- [Mixtral of Experts](https://mlsanity.com/paper/2401.04088/) (2024)
- [GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding](https://mlsanity.com/paper/2006.16668/) (2020)
- [Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time](https://mlsanity.com/paper/2203.05482/) (2022)
- [Editing Models with Task Arithmetic](https://mlsanity.com/paper/2212.04089/) (2022)
- [Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm](https://mlsanity.com/paper/1712.01815/) (2017)
- [Merging Models with Fisher-Weighted Averaging](https://mlsanity.com/paper/2111.09832/) (2021)
- [Generalized End-to-End Loss for Speaker Verification](https://mlsanity.com/paper/1710.10467/) (2017)
- [Branch-Train-Merge: Embarrassingly Parallel Training of Expert Language Models](https://mlsanity.com/paper/2208.03306/) (2022)
- [The Illusion of State in State-Space Models](https://mlsanity.com/paper/2404.08819/) (2024)
- [Acquisition of Chess Knowledge in AlphaZero](https://mlsanity.com/paper/2111.09259/) (2021)
- [Model merging with SVD to tie the Knots](https://mlsanity.com/paper/2410.19735/) (2024)
- [Aligning Superhuman AI with Human Behavior: Chess as a Model System](https://mlsanity.com/paper/2006.01855/) (2020)
- [ChessGPT: Bridging Policy Learning and Language Modeling](https://mlsanity.com/paper/2306.09200/) (2023)
- [Chess as a Testbed for Language Model State Tracking](https://mlsanity.com/paper/2102.13249/) (2021)
- [Learning to Route Among Specialized Experts for Zero-Shot Generalization](https://mlsanity.com/paper/2402.05859/) (2024)
- [Emergent World Models and Latent Variable Estimation in Chess-Playing Language Models](https://mlsanity.com/paper/2403.15498/) (2024)
- [Amortized Planning with Large-Scale Transformers: A Case Study on Chess](https://mlsanity.com/paper/2402.04494/) (2024)
- [The Chess Transformer: Mastering Play using Generative Language Models](https://mlsanity.com/paper/2008.04057/) (2020)
- [Evidence of Learned Look-Ahead in a Chess-Playing Neural Network](https://mlsanity.com/paper/2406.00877/) (2024)
- [The Shrinking Landscape of Linguistic Diversity in the Age of Large Language Models](https://mlsanity.com/paper/2502.11266/) (2025)
- [Maia-2: A Unified Model for Human-AI Alignment in Chess](https://mlsanity.com/paper/2409.20553/) (2024)
- [Learning Models of Individual Behavior in Chess](https://mlsanity.com/paper/2008.10086/) (2020)
- [Parameter-Efficient Sparsity Crafting from Dense to Mixture-of-Experts for Instruction Tuning on General Tasks](https://mlsanity.com/paper/2401.02731/) (2024)
- [Transcendence: Generative Models Can Outperform The Experts That Train Them](https://mlsanity.com/paper/2406.11741/) (2024)
- [Explore the Reasoning Capability of LLMs in the Chess Testbed](https://mlsanity.com/paper/2411.06655/) (2024)
- [Detecting Individual Decision-Making Style: Exploring Behavioral Stylometry in Chess](https://mlsanity.com/paper/2208.01366/) (2022)

## Built on by (most cited first, 1 of 1)

- [Towards Generalist Game Players: An Investigation of Foundation Models in the Game Multiverse](https://mlsanity.com/paper/2605.09965/) (2026)

## Abstract (arXiv)

Modern chess language models are dense transformers trained on millions of games played by thousands of high-rated individuals. However, these monolithic networks tend to collapse into mode-averaged behavior, where stylistic boundaries are blurred, and rare but effective strategies are suppressed. To counteract homogenization, we introduce Mixture-of-Masters (MoM), the first chess mixture-of-experts model with small-sized GPT experts emulating world-class grandmasters. For each move, a post-hoc learnable gating network selects the most appropriate persona to channel depending on the game state, allowing MoM to switch its style dynamically, e.g., Tal's offensive vocation or Petrosian's defensive solidity. When evaluated against Stockfish on unseen standard games, MoM outperforms both dense individual expert networks and popular GPT baselines trained on aggregated data, while ensuring generation variety, control, and interpretability.