# Visual Encoders for Data-Efficient Imitation Learning in Modern Video Games

arXiv: https://arxiv.org/abs/2312.02312 · HTML: https://mlsanity.com/paper/2312.02312/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 32 of its 42 references that are in the map.
- First posted on arXiv 4 Dec 2023.

## Placed in

- Offline RL and imitation learning from data / [Imitation learning methods](https://mlsanity.com/offlinerl/imitation-learning-methods/)

## Builds on (most cited first, 30 of 32)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [Voyager: An Open-Ended Embodied Agent with Large Language Models](https://mlsanity.com/paper/2305.16291/) (2023)
- [Identity Mappings in Deep Residual Networks](https://mlsanity.com/paper/1603.05027/) (2016)
- [Dota 2 with Large Scale Deep Reinforcement Learning](https://mlsanity.com/paper/1912.06680/) (2019)
- [IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures](https://mlsanity.com/paper/1802.01561/) (2018)
- [A Generalist Agent](https://mlsanity.com/paper/2205.06175/) (2022)
- [R3M: A Universal Visual Representation for Robot Manipulation](https://mlsanity.com/paper/2203.12601/) (2022)
- [How to train your ViT? Data, Augmentation, and Regularization in Vision Transformers](https://mlsanity.com/paper/2106.10270/) (2021)
- [MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledge](https://mlsanity.com/paper/2206.08853/) (2022)
- [Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videos](https://mlsanity.com/paper/2206.11795/) (2022)
- [Imitating Human Behaviour with Diffusion Models](https://mlsanity.com/paper/2301.10677/) (2023)
- [MineRL: A Large-Scale Dataset of Minecraft Demonstrations](https://mlsanity.com/paper/1907.13440/) (2019)
- [Focal Modulation Networks](https://mlsanity.com/paper/2203.11926/) (2022)
- [STEVE-1: A Generative Model for Text-to-Behavior in Minecraft](https://mlsanity.com/paper/2306.00937/) (2023)
- [Pre-Trained Image Encoder for Generalizable Visual Reinforcement Learning](https://mlsanity.com/paper/2212.08860/) (2022)
- [Counter-Strike Deathmatch with Large-Scale Behavioural Cloning](https://mlsanity.com/paper/2104.04258/) (2021)
- [GROOT: Learning to Follow Instructions by Watching Gameplay Videos](https://mlsanity.com/paper/2310.08235/) (2023)
- [Open-World Multi-Task Control Through Goal-Aware Representation Learning and Adaptive Horizon Prediction](https://mlsanity.com/paper/2301.10034/) (2023)
- ["It's Unwieldy and It Takes a Lot of Time." Challenges and Opportunities for Creating Agents in Commercial Games](https://mlsanity.com/paper/2009.00541/) (2020)
- [Benchmarking End-to-End Behavioural Cloning on Video Games](https://mlsanity.com/paper/2004.00981/) (2020)
- [Towards Informed Design and Validation Assistance in Computer Games Using Imitation Learning](https://mlsanity.com/paper/2208.07811/) (2022)
- [Technical Challenges of Deploying Reinforcement Learning Agents for Game Testing in AAA Games](https://mlsanity.com/paper/2307.11105/) (2023)

## Built on by (most cited first, 6 of 6)

- [A Survey on Game Playing Agents and Large Models: Methods, Applications, and Challenges](https://mlsanity.com/paper/2403.10249/) (2024)
- [VIEW: Visual Imitation Learning with Waypoints](https://mlsanity.com/paper/2404.17906/) (2024)
- [Pixels to Play: A Foundation Model for 3D Gameplay](https://mlsanity.com/paper/2508.14295/) (2025)
- [Generative Modeling of Individual Behavior at Scale](https://mlsanity.com/paper/2502.14998/) (2025)
- [Pre-trained Visual Representations Generalize Where it Matters in Model-Based Reinforcement Learning](https://mlsanity.com/paper/2509.12531/) (2025)
- [VRScout: Towards Real-Time, Autonomous Testing of Virtual Reality Games](https://mlsanity.com/paper/2511.00002/) (2025)

## Abstract (arXiv)

Video games have served as useful benchmarks for the decision-making community, but going beyond Atari games towards modern games has been prohibitively expensive for the vast majority of the research community. Prior work in modern video games typically relied on game-specific integration to obtain game features and enable online training, or on existing large datasets. An alternative approach is to train agents using imitation learning to play video games purely from images. However, this setting poses a fundamental question: which visual encoders obtain representations that retain information critical for decision making? To answer this question, we conduct a systematic study of imitation learning with publicly available pre-trained visual encoders compared to the typical task-specific end-to-end training approach in Minecraft, Counter-Strike: Global Offensive, and Minecraft Dungeons. Our results show that end-to-end training can be effective with comparably low-resolution images and only minutes of demonstrations, but significant improvements can be gained by utilising pre-trained encoders such as DINOv2 depending on the game. In addition to enabling effective decision making, we show that pre-trained encoders can make decision-making research in video games more accessible by significantly reducing the cost of training.