mlsanityData to 14 Oct 2026

Representation Matters for Mastering Chess: Improved Feature Representation in AlphaZero Outperforms Switching to Transformers

arXiv 2304.14918 ↗First posted 28 Apr 2023cs.AIcited by 8

Not on the map. Why?

Abstract (arXiv)

While transformers have gained recognition as a versatile tool for artificial intelligence (AI), an unexplored challenge arises in the context of chess - a classical AI benchmark. Here, incorporating Vision Transformers (ViTs) into AlphaZero is insufficient for chess mastery, mainly due to ViTs' computational limitations. The attempt to optimize their efficiency by combining MobileNet and NextViT outperformed AlphaZero by about 30 Elo. However, we propose a practical improvement that involves a simple change in the input representation and value loss functions. As a result, we achieve a significant performance boost of up to 180 Elo points beyond what is currently achievable with AlphaZero in chess. In addition to these improvements, our experimental results using the Integrated Gradient technique confirm the effectiveness of the newly introduced features.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 23 of its 34 references that are in the map. First posted on arXiv 28 Apr 2023.

Builds on

23 of 34 references.

References · 23

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
  5. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,795
  6. 2017Axiomatic Attribution for Deep Networkscited by 3,384
  7. 2016Wide Residual Networkscited by 3,070
  8. 2019Searching for MobileNetV3cited by 1,994
  9. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,546
  10. 2021Improving language models by retrieving from trillions of tokenscited by 1,293
  11. 2016Deep Networks with Stochastic Depthcited by 979
  12. 2020A Survey on Visual Transformercited by 654
11 more
  1. 2021Offline Reinforcement Learning as One Big Sequence Modeling Problemcited by 644
  2. 2021LeViT: a Vision Transformer in ConvNet's Clothing for Faster Inferencecited by 475
  3. 2022Multimodal Learning with Transformers: A Surveycited by 400
  4. 2021Chess as a Testbed for Language Model State Trackingcited by 72
  5. 2022Next-ViT: Next Generation Vision Transformer for Efficient Deployment in Realistic Industrial Scenarioscited by 65
  6. 2019Accelerating Self-Play Learning in Gocited by 64
  7. 2023A Survey on Transformers in Reinforcement Learningcited by 50
  8. 2020The Chess Transformer: Mastering Play using Generative Language Modelscited by 37
  9. 2022How Crucial is Transformer in Decision Transformer?cited by 14
  10. 2022Improving Chess Commentaries by Combining Language Models with Symbolic Reasoning Enginescited by 13
  11. 2022Learning Chess With Language Models and Transformerscited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 3; 2025: 3; 2026: 2.

8 papers.

  1. 2024Mastering Chess with a Transformer Modelcited by 15
  2. 2024Checkmating One, by Using Many: Combining Mixture of Experts with MCTS to Improve in Chesscited by 4
  3. 2025Deep Reinforcement Learning via Object-Centric Attentioncited by 2
  4. 2025Towards Piece-by-Piece Explanations for Chess Positions with SHAPcited by 2
  5. 2026ACDZero: MCTS Agent for Mastering Automated Cyber Defensecited by 2
  6. 2024AlphaViT: A flexible game-playing AI for multiple games and variable board sizescited by 0
  7. 2025Understanding the learned look-ahead behavior of chess neural networkscited by 0
  8. 2026WallZero: Mastering the Game of WallGo with Strategic Analysiscited by 0