mlsanityData to 7 Oct 2026

Contrastive Sparse Autoencoders for Interpreting Planning of Chess-Playing Agents

arXiv 2406.04028 ↗First posted 6 Jun 2024cs.AIcited by 2

Not on the map. Why?

Abstract (arXiv)

AI led chess systems to a superhuman level, yet these systems heavily rely on black-box algorithms. This is unsustainable in ensuring transparency to the end-user, particularly when these systems are responsible for sensitive decision-making. Recent interpretability work has shown that the inner representations of Deep Neural Networks (DNNs) were fathomable and contained human-understandable concepts. Yet, these methods are seldom contextualised and are often based on a single hidden state, which makes them unable to interpret multi-step reasoning, e.g. planning. In this respect, we propose contrastive sparse autoencoders (CSAE), a novel framework for studying pairs of game trajectories. Using CSAE, we are able to extract and interpret concepts that are meaningful to the chess-agent plans. We primarily focused on a qualitative analysis of the CSAE features before proposing an automated feature taxonomy. Furthermore, to evaluate the quality of our trained CSAE, we devise sanity checks to wave spurious correlations in our results.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 23 of its 37 references that are in the map. First posted on arXiv 6 Jun 2024.

Builds on

23 of 37 references.

References · 23

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2017Squeeze-and-Excitation Networkscited by 4,763
  3. 2015Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networkscited by 2,048
  4. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,359
  5. 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,309
  6. 2016Understanding intermediate layers using linear classifier probescited by 1,279
  7. 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 733
  8. 2023Towards Automated Circuit Discovery for Mechanistic Interpretabilitycited by 700
  9. 2023Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervisioncited by 445
  10. 2023Finding Neurons in a Haystack: Case Studies with Sparse Probingcited by 328
  11. 2023Emergent Linear Representations in World Models of Self-Supervised Sequence Modelscited by 313
  12. 2020Concept Whitening for Interpretable Image Recognitioncited by 225
11 more
  1. 2024Improving Dictionary Learning with Gated Sparse Autoencoderscited by 171
  2. 2023Linear Representations of Sentiment in Large Language Modelscited by 157
  3. 2021Acquisition of Chess Knowledge in AlphaZerocited by 120
  4. 2024A Multimodal Automated Interpretability Agentcited by 62
  5. 2023Linear Spaces of Meanings: Compositional Structures in Vision-Language Modelscited by 48
  6. 2023Bridging the Human-AI Knowledge Gap: Concept Discovery and Transfer in AlphaZerocited by 41
  7. 2024Learning Interpretable Concepts: Unifying Causal Representation Learning and Foundation Modelscited by 29
  8. 2023From Hope to Safety: Unlearning Biases of Deep Models via Gradient Penalization in Latent Spacecited by 20
  9. 2019Biasing MCTS with Features for General Gamescited by 4
  10. 2022Reinforcement Learning in an Adaptable Chess Environment for Detecting Human-understandable Conceptscited by 3
  11. 2023Information based explanation methods for deep learning agents -- with applications on large open-source chess modelscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Tracing the Thought of a Grandmaster-level Chess-Playing Transformercited by 2
  2. 2026Neuralyzing the Trace: Selective Representation-Level Unlearning with Contrastive Sparse Autoencoderscited by 0