# Towards General Game Representations: Decomposing Games Pixels into Content and Style

arXiv: https://arxiv.org/abs/2307.11141 · HTML: https://mlsanity.com/paper/2307.11141/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 42 references that are in the map.
- First posted on arXiv 20 Jul 2023.

## Builds on (most cited first, 30 of 30)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [Dota 2 with Large Scale Deep Reinforcement Learning](https://mlsanity.com/paper/1912.06680/) (2019)
- [Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm](https://mlsanity.com/paper/1712.01815/) (2017)
- [Deep Visual Domain Adaptation: A Survey](https://mlsanity.com/paper/1802.03601/) (2018)
- [GANSpace: Discovering Interpretable GAN Controls](https://mlsanity.com/paper/2004.02546/) (2020)
- [Zero-Shot Learning -- A Comprehensive Evaluation of the Good, the Bad and the Ugly](https://mlsanity.com/paper/1707.00600/) (2017)
- [Google Research Football: A Novel Reinforcement Learning Environment](https://mlsanity.com/paper/1907.11180/) (2019)
- [ViTAE: Vision Transformer Advanced by Exploring Intrinsic Inductive Bias](https://mlsanity.com/paper/2106.03348/) (2021)
- [Multi-Game Decision Transformers](https://mlsanity.com/paper/2205.15241/) (2022)
- [Neural Style Transfer: A Review](https://mlsanity.com/paper/1705.04058/) (2017)
- [ViZDoom: A Doom-based AI Research Platform for Visual Reinforcement Learning](https://mlsanity.com/paper/1605.02097/) (2016)
- [Unsupervised State Representation Learning in Atari](https://mlsanity.com/paper/1906.08226/) (2019)
- [DriveGAN: Towards a Controllable High-Quality Neural Simulation](https://mlsanity.com/paper/2104.15060/) (2021)
- [Separating Style and Content for Generalized Style Transfer](https://mlsanity.com/paper/1711.06454/) (2017)
- [WarpedGANSpace: Finding non-linear RBF paths in GAN latent space](https://mlsanity.com/paper/2109.13357/) (2021)
- [Style and Content Disentanglement in Generative Adversarial Networks](https://mlsanity.com/paper/1811.05621/) (2018)
- [The Arousal video Game AnnotatIoN (AGAIN) Dataset](https://mlsanity.com/paper/2104.02643/) (2021)
- [PCAAE: Principal Component Analysis Autoencoder for organising the latent space of generative networks](https://mlsanity.com/paper/2006.07827/) (2020)
- [From Pixels to Affect: A Study on Games and Player Experience](https://mlsanity.com/paper/1907.02288/) (2019)
- [Measuring the Biases and Effectiveness of Content-Style Disentanglement](https://mlsanity.com/paper/2008.12378/) (2020)
- [Contrastive Learning of Generalized Game Representations](https://mlsanity.com/paper/2106.10060/) (2021)
- [Go-Blend behavior and affect](https://mlsanity.com/paper/2109.13388/) (2021)
- [Learning Task-Independent Game State Representations from Unlabeled Images](https://mlsanity.com/paper/2206.06490/) (2022)
- [Pretrained Encoders are All You Need](https://mlsanity.com/paper/2106.05139/) (2021)
- [Game State Learning via Game Scene Augmentation](https://mlsanity.com/paper/2207.01289/) (2022)

## Built on by (most cited first, 3 of 3)

- [Visual Encoders for Data-Efficient Imitation Learning in Modern Video Games](https://mlsanity.com/paper/2312.02312/) (2023)
- [BehAVE: Behaviour Alignment of Video Game Encodings](https://mlsanity.com/paper/2402.01335/) (2024)
- [Future Research Avenues for Artificial Intelligence in Digital Gaming: An Exploratory Report](https://mlsanity.com/paper/2412.14085/) (2024)

## Abstract (arXiv)

On-screen game footage contains rich contextual information that players process when playing and experiencing a game. Learning pixel representations of games can benefit artificial intelligence across several downstream tasks including game-playing agents, procedural content generation, and player modelling. The generalizability of these methods, however, remains a challenge, as learned representations should ideally be shared across games with similar game mechanics. This could allow, for instance, game-playing agents trained on one game to perform well in similar games with no re-training. This paper explores how generalizable pre-trained computer vision encoders can be for such tasks, by decomposing the latent space into content embeddings and style embeddings. The goal is to minimize the domain gap between games of the same genre when it comes to game content critical for downstream tasks, and ignore differences in graphical style. We employ a pre-trained Vision Transformer encoder and a decomposition technique based on game genres to obtain separate content and style embeddings. Our findings show that the decomposed embeddings achieve style invariance across multiple games while still maintaining strong content extraction capabilities. We argue that the proposed decomposition of content and style offers better generalization capacities across game environments independently of the downstream task.