# Reinforcement Learning in an Adaptable Chess Environment for Detecting Human-understandable Concepts

arXiv: https://arxiv.org/abs/2211.05500 · HTML: https://mlsanity.com/paper/2211.05500/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 16 references that are in the map.
- First posted on arXiv 10 Nov 2022.

## Builds on (most cited first, 5 of 5)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Understanding intermediate layers using linear classifier probes](https://mlsanity.com/paper/1610.01644/) (2016)
- [Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)](https://mlsanity.com/paper/1711.11279/) (2017)
- [TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems](https://mlsanity.com/paper/1603.04467/) (2016)
- [Acquisition of Chess Knowledge in AlphaZero](https://mlsanity.com/paper/2111.09259/) (2021)

## Built on by (most cited first, 4 of 4)

- [A Review On Safe Reinforcement Learning Using Lyapunov and Barrier Functions](https://mlsanity.com/paper/2508.09128/) (2025)
- [Information based explanation methods for deep learning agents -- with applications on large open-source chess models](https://mlsanity.com/paper/2309.09702/) (2023)
- [Contrastive Sparse Autoencoders for Interpreting Planning of Chess-Playing Agents](https://mlsanity.com/paper/2406.04028/) (2024)
- [Concept backpropagation: An Explainable AI approach for visualising learned concepts in neural network models](https://mlsanity.com/paper/2307.12601/) (2023)

## Abstract (arXiv)

Self-trained autonomous agents developed using machine learning are showing great promise in a variety of control settings, perhaps most remarkably in applications involving autonomous vehicles. The main challenge associated with self-learned agents in the form of deep neural networks, is their black-box nature: it is impossible for humans to interpret deep neural networks. Therefore, humans cannot directly interpret the actions of deep neural network based agents, or foresee their robustness in different scenarios. In this work, we demonstrate a method for probing which concepts self-learning agents internalise in the course of their training. For demonstration, we use a chess playing agent in a fast and light environment developed specifically to be suitable for research groups without access to enormous computational resources or machine learning models.