# Neural Game Engine: Accurate learning of generalizable forward models from pixels

arXiv: https://arxiv.org/abs/2003.10520 · HTML: https://mlsanity.com/paper/2003.10520/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Mar 2020.

## Built on by (most cited first, 12 of 12)

- [Genie: Generative Interactive Environments](https://mlsanity.com/paper/2402.15391/) (2024)
- [Diffusion for World Modeling: Visual Details Matter in Atari](https://mlsanity.com/paper/2405.12399/) (2024)
- [Video as the New Language for Real-World Decision Making](https://mlsanity.com/paper/2402.17139/) (2024)
- [Pandora: Towards General World Model with Natural Language Actions and Video States](https://mlsanity.com/paper/2406.09455/) (2024)
- [Vid2World: Crafting Video Diffusion Models to Interactive World Models](https://mlsanity.com/paper/2505.14357/) (2025)
- [VaViM and VaVAM: Autonomous Driving through Video Generative Modeling](https://mlsanity.com/paper/2502.15672/) (2025)
- [Modeling Worlds in Text](https://mlsanity.com/paper/2106.09578/) (2021)
- [MASS: Multiplayer World Models with Authoritative Shared State](https://mlsanity.com/paper/2608.06257/) (2026)
- [EgoAgent: A Joint Predictive Agent Model in Egocentric Worlds](https://mlsanity.com/paper/2502.05857/) (2025)
- [Evolutionary Planning in Latent Space](https://mlsanity.com/paper/2011.11293/) (2020)
- [Predictive Control Using Learned State Space Models via Rolling Horizon Evolution](https://mlsanity.com/paper/2106.13911/) (2021)
- [TARDIS STRIDE: A Spatio-Temporal Road Image Dataset and World Model for Autonomy](https://mlsanity.com/paper/2506.11302/) (2025)

## Abstract (arXiv)

Access to a fast and easily copied forward model of a game is essential for model-based reinforcement learning and for algorithms such as Monte Carlo tree search, and is also beneficial as a source of unlimited experience data for model-free algorithms. Learning forward models is an interesting and important challenge in order to address problems where a model is not available. Building upon previous work on the Neural GPU, this paper introduces the Neural Game Engine, as a way to learn models directly from pixels. The learned models are able to generalise to different size game levels to the ones they were trained on without loss of accuracy. Results on 10 deterministic General Video Game AI games demonstrate competitive performance, with many of the games models being learned perfectly both in terms of pixel predictions and reward predictions. The pre-trained models are available through the OpenAI Gym interface and are available publicly for future research here: urlhttps://github.com/Bam4d/Neural-Game-Engine