# Evaluation Beyond Task Performance: Analyzing Concepts in AlphaZero in Hex

arXiv: https://arxiv.org/abs/2211.14673 · HTML: https://mlsanity.com/paper/2211.14673/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 87 references that are in the map.
- First posted on arXiv 26 Nov 2022.

## Builds on (most cited first, 30 of 32)

- [Towards A Rigorous Science of Interpretable Machine Learning](https://mlsanity.com/paper/1702.08608/) (2017)
- [Understanding intermediate layers using linear classifier probes](https://mlsanity.com/paper/1610.01644/) (2016)
- [Transformer Feed-Forward Layers Are Key-Value Memories](https://mlsanity.com/paper/2012.14913/) (2020)
- [BERT Rediscovers the Classical NLP Pipeline](https://mlsanity.com/paper/1905.05950/) (2019)
- [Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inference](https://mlsanity.com/paper/1902.01007/) (2019)
- [Probing Classifiers: Promises, Shortcomings, and Advances](https://mlsanity.com/paper/2102.12452/) (2021)
- [Annotation Artifacts in Natural Language Inference Data](https://mlsanity.com/paper/1803.02324/) (2018)
- [Designing and Interpreting Probes with Control Tasks](https://mlsanity.com/paper/1909.03368/) (2019)
- [BLiMP: The Benchmark of Linguistic Minimal Pairs for English](https://mlsanity.com/paper/1912.00582/) (2019)
- [What you can cram into a single vector: Probing sentence embeddings for linguistic properties](https://mlsanity.com/paper/1805.01070/) (2018)
- [What do you learn from context? Probing for sentence structure in contextualized word representations](https://mlsanity.com/paper/1905.06316/) (2019)
- [Assessing the Ability of LSTMs to Learn Syntax-Sensitive Dependencies](https://mlsanity.com/paper/1611.01368/) (2016)
- [Deep Reinforcement Learning for Robotic Manipulation with Asynchronous Off-Policy Updates](https://mlsanity.com/paper/1610.00633/) (2016)
- [What BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language models](https://mlsanity.com/paper/1907.13528/) (2019)
- [Hypothesis Only Baselines in Natural Language Inference](https://mlsanity.com/paper/1805.01042/) (2018)
- [Targeted Syntactic Evaluation of Language Models](https://mlsanity.com/paper/1808.09031/) (2018)
- [Information-Theoretic Probing with Minimum Description Length](https://mlsanity.com/paper/2003.12298/) (2020)
- [A Systematic Assessment of Syntactic Generalization in Neural Language Models](https://mlsanity.com/paper/2005.03692/) (2020)
- [Deep Reinforcement Learning for De-Novo Drug Design](https://mlsanity.com/paper/1711.10907/) (2017)
- [Acquisition of Chess Knowledge in AlphaZero](https://mlsanity.com/paper/2111.09259/) (2021)
- [Scaling Scaling Laws with Board Games](https://mlsanity.com/paper/2104.03113/) (2021)
- [Deep Learning Predicts Hip Fracture using Confounding Patient and Healthcare Variables](https://mlsanity.com/paper/1811.03695/) (2018)
- [Horizon: Facebook's Open Source Applied Reinforcement Learning Platform](https://mlsanity.com/paper/1811.00260/) (2018)
- [Accountability in an Algorithmic Society: Relationality, Responsibility, and Robustness in Machine Learning](https://mlsanity.com/paper/2202.05338/) (2022)
- [ELF OpenGo: An Analysis and Open Reimplementation of AlphaZero](https://mlsanity.com/paper/1902.04522/) (2019)
- [Natural Environment Benchmarks for Reinforcement Learning](https://mlsanity.com/paper/1811.06032/) (2018)
- [Sorting through the noise: Testing robustness of information processing in pre-trained language models](https://mlsanity.com/paper/2109.12393/) (2021)
- [Measuring and Characterizing Generalization in Deep Reinforcement Learning](https://mlsanity.com/paper/1812.02868/) (2018)
- [Hyperparameter Optimization Is Deceiving Us, and How to Stop It](https://mlsanity.com/paper/2102.03034/) (2021)
- [Model Selection's Disparate Impact in Real-World Deep Learning Applications](https://mlsanity.com/paper/2104.00606/) (2021)

## Built on by (most cited first, 8 of 8)

- [Emergent Linear Representations in World Models of Self-Supervised Sequence Models](https://mlsanity.com/paper/2309.00941/) (2023)
- [Sudden Drops in the Loss: Syntax Acquisition, Phase Transitions, and Simplicity Bias in MLMs](https://mlsanity.com/paper/2309.07311/) (2023)
- [We Can't Understand AI Using our Existing Vocabulary](https://mlsanity.com/paper/2502.07586/) (2025)
- [Hidden Breakthroughs in Language Model Training](https://mlsanity.com/paper/2506.15872/) (2025)
- [AlphaZero Neural Scaling and Zipf's Law: a Tale of Board Games and Power Laws](https://mlsanity.com/paper/2412.11979/) (2024)
- [INSPECT: Intrinsic and Systematic Probing Evaluation for Code Transformers](https://mlsanity.com/paper/2312.05092/) (2023)
- [The Deleuzian Representation Hypothesis](https://mlsanity.com/paper/2512.19734/) (2025)
- [Belief-Guided Decision Making with Uncertainty Gating in the Game of Go](https://mlsanity.com/paper/2607.26946/) (2026)

## Abstract (arXiv)

AlphaZero, an approach to reinforcement learning that couples neural networks and Monte Carlo tree search (MCTS), has produced state-of-the-art strategies for traditional board games like chess, Go, shogi, and Hex. While researchers and game commentators have suggested that AlphaZero uses concepts that humans consider important, it is unclear how these concepts are captured in the network. We investigate AlphaZero's internal representations in the game of Hex using two evaluation techniques from natural language processing (NLP): model probing and behavioral tests. In doing so, we introduce new evaluation tools to the RL community and illustrate how evaluations other than task performance can be used to provide a more complete picture of a model's strengths and weaknesses. Our analyses in the game of Hex reveal interesting patterns and generate some testable hypotheses about how such models learn in general. For example, we find that MCTS discovers concepts before the neural network learns to encode them. We also find that concepts related to short-term end-game planning are best encoded in the final layers of the model, whereas concepts related to long-term planning are encoded in the middle layers of the model.