# Contrastive Sparse Autoencoders for Interpreting Planning of Chess-Playing Agents

arXiv: https://arxiv.org/abs/2406.04028 · HTML: https://mlsanity.com/paper/2406.04028/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Steering and interpreting LLM reasoning, one of 13 directions in Mechanistic interpretability.
- Builds on 23 of its 37 references that are in the map.
- First posted on arXiv 6 Jun 2024.

## Placed in

- Mechanistic interpretability / [Steering and interpreting LLM reasoning](https://mlsanity.com/mechinterp/steering-and-interpreting-llm-reasoning/)

## Builds on (most cited first, 23 of 23)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks](https://mlsanity.com/paper/1511.06434/) (2015)
- [Sparse Autoencoders Find Highly Interpretable Features in Language Models](https://mlsanity.com/paper/2309.08600/) (2023)
- [Understanding intermediate layers using linear classifier probes](https://mlsanity.com/paper/1610.01644/) (2016)
- [Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)](https://mlsanity.com/paper/1711.11279/) (2017)
- [Discovering Latent Knowledge in Language Models Without Supervision](https://mlsanity.com/paper/2212.03827/) (2022)
- [Towards Automated Circuit Discovery for Mechanistic Interpretability](https://mlsanity.com/paper/2304.14997/) (2023)
- [Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervision](https://mlsanity.com/paper/2312.09390/) (2023)
- [Finding Neurons in a Haystack: Case Studies with Sparse Probing](https://mlsanity.com/paper/2305.01610/) (2023)
- [Emergent Linear Representations in World Models of Self-Supervised Sequence Models](https://mlsanity.com/paper/2309.00941/) (2023)
- [Concept Whitening for Interpretable Image Recognition](https://mlsanity.com/paper/2002.01650/) (2020)
- [Improving Dictionary Learning with Gated Sparse Autoencoders](https://mlsanity.com/paper/2404.16014/) (2024)
- [Linear Representations of Sentiment in Large Language Models](https://mlsanity.com/paper/2310.15154/) (2023)
- [Acquisition of Chess Knowledge in AlphaZero](https://mlsanity.com/paper/2111.09259/) (2021)
- [A Multimodal Automated Interpretability Agent](https://mlsanity.com/paper/2404.14394/) (2024)
- [Linear Spaces of Meanings: Compositional Structures in Vision-Language Models](https://mlsanity.com/paper/2302.14383/) (2023)
- [Bridging the Human-AI Knowledge Gap: Concept Discovery and Transfer in AlphaZero](https://mlsanity.com/paper/2310.16410/) (2023)
- [Learning Interpretable Concepts: Unifying Causal Representation Learning and Foundation Models](https://mlsanity.com/paper/2402.09236/) (2024)
- [From Hope to Safety: Unlearning Biases of Deep Models via Gradient Penalization in Latent Space](https://mlsanity.com/paper/2308.09437/) (2023)
- [Biasing MCTS with Features for General Games](https://mlsanity.com/paper/1903.08942/) (2019)
- [Reinforcement Learning in an Adaptable Chess Environment for Detecting Human-understandable Concepts](https://mlsanity.com/paper/2211.05500/) (2022)
- [Information based explanation methods for deep learning agents -- with applications on large open-source chess models](https://mlsanity.com/paper/2309.09702/) (2023)

## Built on by (most cited first, 2 of 2)

- [Tracing the Thought of a Grandmaster-level Chess-Playing Transformer](https://mlsanity.com/paper/2604.10158/) (2026)
- [Neuralyzing the Trace: Selective Representation-Level Unlearning with Contrastive Sparse Autoencoders](https://mlsanity.com/paper/2609.31056/) (2026)

## Abstract (arXiv)

AI led chess systems to a superhuman level, yet these systems heavily rely on black-box algorithms. This is unsustainable in ensuring transparency to the end-user, particularly when these systems are responsible for sensitive decision-making. Recent interpretability work has shown that the inner representations of Deep Neural Networks (DNNs) were fathomable and contained human-understandable concepts. Yet, these methods are seldom contextualised and are often based on a single hidden state, which makes them unable to interpret multi-step reasoning, e.g. planning. In this respect, we propose contrastive sparse autoencoders (CSAE), a novel framework for studying pairs of game trajectories. Using CSAE, we are able to extract and interpret concepts that are meaningful to the chess-agent plans. We primarily focused on a qualitative analysis of the CSAE features before proposing an automated feature taxonomy. Furthermore, to evaluate the quality of our trained CSAE, we devise sanity checks to wave spurious correlations in our results.