# Contrastive Learning of Generalized Game Representations

arXiv: https://arxiv.org/abs/2106.10060 · HTML: https://mlsanity.com/paper/2106.10060/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 38 references that are in the map.
- First posted on arXiv 18 Jun 2021.

## Builds on (most cited first, 24 of 24)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [World Models](https://mlsanity.com/paper/1803.10122/) (2018)
- [CURL: Contrastive Unsupervised Representations for Reinforcement Learning](https://mlsanity.com/paper/2004.04136/) (2020)
- [Deep Visual Domain Adaptation: A Survey](https://mlsanity.com/paper/1802.03601/) (2018)
- [Reinforcement Learning with Augmented Data](https://mlsanity.com/paper/2004.14990/) (2020)
- [Agent57: Outperforming the Atari Human Benchmark](https://mlsanity.com/paper/2003.13350/) (2020)
- [Google Research Football: A Novel Reinforcement Learning Environment](https://mlsanity.com/paper/1907.11180/) (2019)
- [Representation Learning via Invariant Causal Mechanisms](https://mlsanity.com/paper/2010.07922/) (2020)
- [ViZDoom: A Doom-based AI Research Platform for Visual Reinforcement Learning](https://mlsanity.com/paper/1605.02097/) (2016)
- [Generalization in Deep Learning](https://mlsanity.com/paper/1710.05468/) (2017)
- [Learning to Simulate Dynamic Environments with GameGAN](https://mlsanity.com/paper/2005.12126/) (2020)
- [Deep Learning for Procedural Content Generation](https://mlsanity.com/paper/2010.04548/) (2020)
- [Out of Distribution Generalization in Machine Learning](https://mlsanity.com/paper/2103.02667/) (2021)
- [The Pixels and Sounds of Emotion: General-Purpose Representations of Arousal in Games](https://mlsanity.com/paper/2101.10706/) (2021)
- [From Pixels to Affect: A Study on Games and Player Experience](https://mlsanity.com/paper/1907.02288/) (2019)
- [Entity Embedding as Game Representation](https://mlsanity.com/paper/2010.01685/) (2020)
- [Toward Game Level Generation from Gameplay Videos](https://mlsanity.com/paper/1602.07721/) (2016)
- [Deep Unsupervised Multi-View Detection of Video Game Stream Highlights](https://mlsanity.com/paper/1807.09715/) (2018)
- [Player Experience Extraction from Gameplay Video](https://mlsanity.com/paper/1809.06201/) (2018)

## Built on by (most cited first, 11 of 11)

- [CLIP meets GamePhysics: Towards bug identification in gameplay videos using zero-shot transfer learning](https://mlsanity.com/paper/2203.11096/) (2022)
- [Domain Generalization -- A Causal Perspective](https://mlsanity.com/paper/2209.15177/) (2022)
- [Affective Game Computing: A Survey](https://mlsanity.com/paper/2309.14104/) (2023)
- [Learning Task-Independent Game State Representations from Unlabeled Images](https://mlsanity.com/paper/2206.06490/) (2022)
- [BehAVE: Behaviour Alignment of Video Game Encodings](https://mlsanity.com/paper/2402.01335/) (2024)
- [Text-Guided Image Clustering](https://mlsanity.com/paper/2402.02996/) (2024)
- [Towards General Game Representations: Decomposing Games Pixels into Content and Style](https://mlsanity.com/paper/2307.11141/) (2023)
- [Can Large Language Models Capture Video Game Engagement?](https://mlsanity.com/paper/2502.04379/) (2025)
- [PartitionVAE -- a human-interpretable VAE](https://mlsanity.com/paper/2302.03689/) (2023)
- [Future Research Avenues for Artificial Intelligence in Digital Gaming: An Exploratory Report](https://mlsanity.com/paper/2412.14085/) (2024)
- [Game-invariant Features Through Contrastive and Domain-adversarial Learning](https://mlsanity.com/paper/2505.17328/) (2025)

## Abstract (arXiv)

Representing games through their pixels offers a promising approach for building general-purpose and versatile game models. While games are not merely images, neural network models trained on game pixels often capture differences of the visual style of the image rather than the content of the game. As a result, such models cannot generalize well even within similar games of the same genre. In this paper we build on recent advances in contrastive learning and showcase its benefits for representation learning in games. Learning to contrast images of games not only classifies games in a more efficient manner; it also yields models that separate games in a more meaningful fashion by ignoring the visual style and focusing, instead, on their content. Our results in a large dataset of sports video games containing 100k images across 175 games and 10 game genres suggest that contrastive learning is better suited for learning generalized game representations compared to conventional supervised learning. The findings of this study bring us closer to universal visual encoders for games that can be reused across previously unseen games without requiring retraining or fine-tuning.