# RePAIR: Predictive Self-Supervised Representation Learning in Chess

arXiv: https://arxiv.org/abs/2606.11860 · HTML: https://mlsanity.com/paper/2606.11860/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 20 references that are in the map.
- First posted on arXiv 10 Jun 2026.

## Builds on (most cited first, 14 of 14)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021)
- [Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture](https://mlsanity.com/paper/2301.08243/) (2023)
- [Lessons on Parameter Sharing across Layers in Transformers](https://mlsanity.com/paper/2104.06022/) (2021)
- [Recurrent Stacking of Layers for Compact Neural Machine Translation Models](https://mlsanity.com/paper/1807.05353/) (2018)
- [Maia-2: A Unified Model for Human-AI Alignment in Chess](https://mlsanity.com/paper/2409.20553/) (2024)
- [Variance-Covariance Regularization Improves Representation Learning](https://mlsanity.com/paper/2306.13292/) (2023)
- [ProjectionPathExplorer: Exploring Visual Patterns in Projected Decision-Making Paths](https://mlsanity.com/paper/2001.08372/) (2020)
- [Learning to Plan via Supervised Contrastive Learning and Strategic Interpolation: A Chess Case Study](https://mlsanity.com/paper/2506.04892/) (2025)
- [Chess2vec: Learning Vector Representations for Chess](https://mlsanity.com/paper/2011.01014/) (2020)

## Abstract (arXiv)

In this paper, we introduce Representation Prediction via Autoencoding using Iterative Refinement (RePAIR) - a novel self-supervised representation learning architecture that synthesizes Masked Autoencoders (MAE), Joint Embedding Predictive Architectures (JEPA), and Bidirectional Encoder Representations from Transformers (BERT). We demonstrate how it can be used to encode objects in sequential data like consecutive chess positions into compact yet meaningful representations. The basic principle of the architecture is to mask large portions of a sequence of latent states, similar to BERT and MAE. Then, we apply a lightweight Predictor to the latent representations that repairs gaps in the sequence in a lower-dimensional embedding space akin to JEPA. Our experiments in the domain of chess show that the Encoder refines the board representations such that meaningful chess concepts emerge clustered in the latent space. Furthermore, reconstructions of the masked board states show that the model is able to reason about the piece movements without relying on costly reinforcement learning methods. Lastly, we find that the resulting representation space allows for quick and intuitive dissections of chess games by observing the game path trajectories in this semantically rich space.