What Counts as Strategic Reasoning? A Systematic Mapping of Chess Research on Humans, Engines, and Language Models
On the map: LLM reasoning → Reasoning for agents, planning and code
Abstract (arXiv)
Chess has long served as a model domain for studying search, expertise, decision-making, and artificial intelligence. The emergence of large language models (LLMs) has renewed the relevance of chess as a controlled environment for investigating strategic reasoning and comparing human and artificial decision-making. We present a systematic mapping study of recent research spanning human players, classical chess engines, neural and reinforcement-learning systems, LLMs, and hybrid approaches. The final map comprises 84 core study families, classified according to agent type, strategic-reasoning stages, and evaluation dimensions. The map reveals a literature strongly concentrated on situation assessment, evaluation, and action selection, while explicit planning, explanation, metacognition, and human--AI collaboration remain less explored. LLM research places particular emphasis on state representation and generalization, whereas grounded explanation appears more frequently in hybrid approaches combining language models with engines, expert knowledge, or other external structures. Two distinctions emerge that the map aggregates rather than resolves: hybrid systems differ in where and when heterogeneous capabilities combine, and evaluations that show improved human performance do not thereby establish human--AI synergy. We propose both as extensions of the mapping framework. We argue that chess provides a useful bridge between cognitive and computational perspectives on strategic reasoning, and identify explicit planning, grounded and faithful explanation, metacognitive calibration, and human--AI complementarity as directions for future research.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 8 of its 33 references that are in the map. First posted on arXiv 16 Sep 2026.
Builds on
8 of 33 references.
References · 8
- 2024When combinations of humans and AI are useful: A systematic review and meta-analysiscited by 106
- 2020Aligning Superhuman AI with Human Behavior: Chess as a Model Systemcited by 79
- 2023ChessGPT: Bridging Policy Learning and Language Modelingcited by 75
- 2024Emergent World Models and Latent Variable Estimation in Chess-Playing Language Modelscited by 62
- 2024Amortized Planning with Large-Scale Transformers: A Case Study on Chesscited by 49
- 2024Evidence of Learned Look-Ahead in a Chess-Playing Neural Networkcited by 34
- 2024Maia-2: A Unified Model for Human-AI Alignment in Chesscited by 25
- 2025Human-Artificial Interaction in the Age of Agentic AI: A System-Theoretical Approachcited by 22
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025ChessArena: A Chess Testbed for Evaluating Strategic Reasoning Capabilities of Large Language Models
- 2024Explore the Reasoning Capability of LLMs in the Chess Testbed
- 2023Large Language Models on the Chessboard: A Study on ChatGPT's Formal Language Comprehension and Complex Reasoning Skills
- 2024Mastering Board Games by External and Internal Planning with Language Models
- 2025LLM CHESS: Benchmarking Reasoning and Instruction-Following in LLMs through Chess
- 2025Xiangqi-R1: Enhancing Spatial Strategic Reasoning in LLMs for Chinese Chess via Reinforcement Learning
- 2025Evaluating Language Models' Evaluations of Games
- 2023Strategic Reasoning with Language Models