# Augmentations for Robust and Efficient Imitation Learning in Streamed Video Games

arXiv: https://arxiv.org/abs/2607.14200 · HTML: https://mlsanity.com/paper/2607.14200/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 14 of its 19 references that are in the map.
- First posted on arXiv 15 Jul 2026.

## Placed in

- Offline RL and imitation learning from data / [Imitation learning methods](https://mlsanity.com/offlinerl/imitation-learning-methods/)

## Builds on (most cited first, 14 of 14)

- [Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World](https://mlsanity.com/paper/1703.06907/) (2017)
- [Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixels](https://mlsanity.com/paper/2004.13649/) (2020)
- [Learning Universal Policies via Text-Guided Video Generation](https://mlsanity.com/paper/2302.00111/) (2023)
- [Reinforcement Learning with Augmented Data](https://mlsanity.com/paper/2004.14990/) (2020)
- [An Algorithmic Perspective on Imitation Learning](https://mlsanity.com/paper/1811.06711/) (2018)
- [Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videos](https://mlsanity.com/paper/2206.11795/) (2022)
- [Imitating Human Behaviour with Diffusion Models](https://mlsanity.com/paper/2301.10677/) (2023)
- [Predictive Inverse Dynamics Models are Scalable Learners for Robotic Manipulation](https://mlsanity.com/paper/2412.15109/) (2024)
- [Theia: Distilling Diverse Vision Foundation Models for Robot Learning](https://mlsanity.com/paper/2407.20179/) (2024)
- [Counter-Strike Deathmatch with Large-Scale Behavioural Cloning](https://mlsanity.com/paper/2104.04258/) (2021)
- [CCIL: Continuity-based Data Augmentation for Corrective Imitation Learning](https://mlsanity.com/paper/2310.12972/) (2023)
- [Latent Diffusion Planning for Imitation Learning](https://mlsanity.com/paper/2504.16925/) (2025)
- [Improving Generalization in Game Agents with Data Augmentation in Imitation Learning](https://mlsanity.com/paper/2309.12815/) (2023)
- [When Does Predictive Inverse Dynamics Outperform Behavior Cloning?](https://mlsanity.com/paper/2601.21718/) (2026)

## Abstract (arXiv)

Imitation learning is an appealing way to scale game-playing agents to complex 3D environments by training policies to map visual observations to actions from human demonstrations. However, these demonstrations are expensive to collect and modern game-playing is often done through streaming in which network delay and compression introduce spatiotemporally correlated visual artifacts that can cause a covariance shift at test time. To address these challenges, we propose streaming augmentations that mimic four types of artifacts commonly encountered during streaming with low-bandwidth network connection: pixelated blocks and scrubs, global blur, and ghosting. We instantiate our approach on top of predictive inverse dynamics models (PIDM), which combine future-state conditioning with an inverse dynamics policy in a learned latent space, and evaluate the impact of our augmentations across three tasks in modern 3D video games. Under stable streaming conditions, agents trained with spatiotemporal augmentations achieve up to 41% higher evaluation performance compared to agents trained without augmentations under an identical data budget. When network lag is introduced, agents trained with augmentations degrade by only 7.45% vs 49.82% of the original performance for agents trained only with the original data. These results clearly indicate that spatiotemporal augmentations tailored for the streaming setting are a simple yet powerful tool to train robust and efficient game-playing agents.