# Improving Chess Commentaries by Combining Language Models with Symbolic Reasoning Engines

arXiv: https://arxiv.org/abs/2212.08195 · HTML: https://mlsanity.com/paper/2212.08195/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning.
- Builds on 8 of its 28 references that are in the map.
- First posted on arXiv 15 Dec 2022.

## Placed in

- LLM reasoning / [Reasoning for agents, planning and code](https://mlsanity.com/lr/reasoning-for-agents-planning-and-code/)

## Builds on (most cited first, 8 of 8)

- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [OPT: Open Pre-trained Transformer Language Models](https://mlsanity.com/paper/2205.01068/) (2022)
- [Scaling Language Models: Methods, Analysis & Insights from Training Gopher](https://mlsanity.com/paper/2112.11446/) (2021)
- [CTRL: A Conditional Transformer Language Model for Controllable Generation](https://mlsanity.com/paper/1909.05858/) (2019)
- [ParlAI: A Dialog Research Software Platform](https://mlsanity.com/paper/1705.06476/) (2017)
- [Acquisition of Chess Knowledge in AlphaZero](https://mlsanity.com/paper/2111.09259/) (2021)
- [Chess as a Testbed for Language Model State Tracking](https://mlsanity.com/paper/2102.13249/) (2021)
- [Controlling Style in Generated Dialogue](https://mlsanity.com/paper/2009.10855/) (2020)

## Built on by (most cited first, 13 of 13)

- [Maia-2: A Unified Model for Human-AI Alignment in Chess](https://mlsanity.com/paper/2409.20553/) (2024)
- [Large Language Model Programs](https://mlsanity.com/paper/2305.05364/) (2023)
- [Representation Matters for Mastering Chess: Improved Feature Representation in AlphaZero Outperforms Switching to Transformers](https://mlsanity.com/paper/2304.14918/) (2023)
- [ChessQA: Evaluating Large Language Models for Chess Understanding](https://mlsanity.com/paper/2510.23948/) (2025)
- [Has It All Been Solved? Open NLP Research Questions Not Solved by Large Language Models](https://mlsanity.com/paper/2305.12544/) (2023)
- [Bridging the Gap between Expert and Language Models: Concept-guided Chess Commentary Generation and Evaluation](https://mlsanity.com/paper/2410.20811/) (2024)
- [From Multimodal Perception to Strategic Reasoning: A Survey on AI-Generated Game Commentary](https://mlsanity.com/paper/2506.17294/) (2025)
- [Mixture of Masters: Sparse Chess Language Models with Player Routing](https://mlsanity.com/paper/2602.04447/) (2026)
- [Exploring Collaboration between a language and a non-language agent](https://mlsanity.com/paper/2609.00474/) (2026)
- [Communicating Chess Strategies in Natural Language](https://mlsanity.com/paper/2607.11486/) (2026)
- [Hallucinations on the Board: Tool-Augmented Evaluation of LLM Chess Commentary](https://mlsanity.com/paper/2608.04240/) (2026)
- [Language Models that Play Chess and Explain Their Moves](https://mlsanity.com/paper/2610.03695/) (2026)
- [GameCommBench: A Unified Benchmark and Type-Aware Evaluation for AI-Generated Game Commentary](https://mlsanity.com/paper/2610.11129/) (2026)

## Abstract (arXiv)

Despite many recent advancements in language modeling, state-of-the-art language models lack grounding in the real world and struggle with tasks involving complex reasoning. Meanwhile, advances in the symbolic reasoning capabilities of AI have led to systems that outperform humans in games like chess and Go (Silver et al., 2018). Chess commentary provides an interesting domain for bridging these two fields of research, as it requires reasoning over a complex board state and providing analyses in natural language. In this work we demonstrate how to combine symbolic reasoning engines with controllable language models to generate chess commentaries. We conduct experiments to demonstrate that our approach generates commentaries that are preferred by human judges over previous baselines.