# Aligning Superhuman AI with Human Behavior: Chess as a Model System

arXiv: https://arxiv.org/abs/2006.01855 · HTML: https://mlsanity.com/paper/2006.01855/

## Key facts

- Cited by 79 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Jun 2020.

## Built on by (most cited first, 30 of 79)

- [Acquisition of Chess Knowledge in AlphaZero](https://mlsanity.com/paper/2111.09259/) (2021)
- [Can You Learn an Algorithm? Generalizing from Easy to Hard Problems with Recurrent Networks](https://mlsanity.com/paper/2106.04537/) (2021)
- [ChessGPT: Bridging Policy Learning and Language Modeling](https://mlsanity.com/paper/2306.09200/) (2023)
- [Modeling Strong and Human-Like Gameplay with KL-Regularized Search](https://mlsanity.com/paper/2112.07544/) (2021)
- [Imitation Learning by Estimating Expertise of Demonstrators](https://mlsanity.com/paper/2202.01288/) (2022)
- [Amortized Planning with Large-Scale Transformers: A Case Study on Chess](https://mlsanity.com/paper/2402.04494/) (2024)
- [Reinforcement Learning for Education: Opportunities and Challenges](https://mlsanity.com/paper/2107.08828/) (2021)
- [Maia-2: A Unified Model for Human-AI Alignment in Chess](https://mlsanity.com/paper/2409.20553/) (2024)
- [Transcendence: Generative Models Can Outperform The Experts That Train Them](https://mlsanity.com/paper/2406.11741/) (2024)
- [Detecting Individual Decision-Making Style: Exploring Behavioral Stylometry in Chess](https://mlsanity.com/paper/2208.01366/) (2022)
- [Reinforcement Learning Under Algorithmic Triage](https://mlsanity.com/paper/2109.11328/) (2021)
- [Improving Human Sequential Decision-Making with Reinforcement Learning](https://mlsanity.com/paper/2108.08454/) (2021)
- [Mimetic Models: Ethical Implications of AI that Acts Like You](https://mlsanity.com/paper/2207.09394/) (2022)
- [Human-aligned Chess with a Bit of Search](https://mlsanity.com/paper/2410.03893/) (2024)
- [Measuring the Non-Transitivity in Chess](https://mlsanity.com/paper/2110.11737/) (2021)
- [Designing Skill-Compatible AI: Methodologies and Frameworks in Chess](https://mlsanity.com/paper/2405.05066/) (2024)
- [From {Solution Synthesis} to {Student Attempt Synthesis} for Block-Based Visual Programming Tasks](https://mlsanity.com/paper/2205.01265/) (2022)
- [Optimal Behavior Prior: Data-Efficient Human Models for Improved Human-AI Collaboration](https://mlsanity.com/paper/2211.01602/) (2022)
- [A Rubric for Human-like Agents and NeuroAI](https://mlsanity.com/paper/2212.04401/) (2022)
- [ChessQA: Evaluating Large Language Models for Chess Understanding](https://mlsanity.com/paper/2510.23948/) (2025)
- [Learning to Imitate with Less: Efficient Individual Behavior Modeling in Chess](https://mlsanity.com/paper/2507.21488/) (2025)
- [Chessformer: A Unified Architecture for Chess Modeling](https://mlsanity.com/paper/2605.19091/) (2026)
- [Trucks Don't Mean Trump: Diagnosing Human Error in Image Analysis](https://mlsanity.com/paper/2205.07333/) (2022)
- [Checkmating One, by Using Many: Combining Mixture of Experts with MCTS to Improve in Chess](https://mlsanity.com/paper/2401.16852/) (2024)
- [Towards Generalizable Agents in Text-Based Educational Environments: A Study of Integrating RL with LLMs](https://mlsanity.com/paper/2404.18978/) (2024)
- [Policy Learning with a Language Bottleneck](https://mlsanity.com/paper/2405.04118/) (2024)
- [Large Language Model Agents for Improving Engagement with Behavior Change Interventions: Application to Digital Mindfulness](https://mlsanity.com/paper/2407.13067/) (2024)
- [Chess Rating Estimation from Moves and Clock Times Using a CNN-LSTM](https://mlsanity.com/paper/2409.11506/) (2024)
- [Weak-to-Strong Generalization beyond Accuracy: a Pilot Study in Safety, Toxicity, and Legal Reasoning](https://mlsanity.com/paper/2410.12621/) (2024)
- [Learning to Plan via Supervised Contrastive Learning and Strategic Interpolation: A Chess Case Study](https://mlsanity.com/paper/2506.04892/) (2025)

## Abstract (arXiv)

As artificial intelligence becomes increasingly intelligent---in some cases, achieving superhuman performance---there is growing potential for humans to learn from and collaborate with algorithms. However, the ways in which AI systems approach problems are often different from the ways people do, and thus may be uninterpretable and hard to learn from. A crucial step in bridging this gap between human and artificial intelligence is modeling the granular actions that constitute human behavior, rather than simply matching aggregate human performance. We pursue this goal in a model system with a long history in artificial intelligence: chess. The aggregate performance of a chess player unfolds as they make decisions over the course of a game. The hundreds of millions of games played online by players at every skill level form a rich source of data in which these decisions, and their exact context, are recorded in minute detail. Applying existing chess engines to this data, including an open-source implementation of AlphaZero, we find that they do not predict human moves well. We develop and introduce Maia, a customized version of Alpha-Zero trained on human chess games, that predicts human moves at a much higher accuracy than existing engines, and can achieve maximum accuracy when predicting decisions made by players at a specific skill level in a tuneable way. For a dual task of predicting whether a human will make a large mistake on the next move, we develop a deep neural network that significantly outperforms competitive baselines. Taken together, our results suggest that there is substantial promise in designing artificial intelligence systems with human collaboration in mind by first accurately modeling granular human decision-making.