Towards Generalist Game Players: An Investigation of Foundation Models in the Game Multiverse
Not on the map. Why?
Abstract (arXiv)
The real world unfolds along a single set of physics laws, yet human intelligence demonstrates a remarkable capacity to generalize experiences from this singular physical existence into a multiverse of games, each governed by entirely different rules, aesthetics, physics, and objectives. This omni-reality adaptability is a hallmark of general intelligence. As Artificial Intelligence progresses towards Artificial General Intelligence, the multiverse of games has evolved from mere entertainment into the ultimate ground for training and evaluating AGI. The pursuit of this generality has unfolded across four eras: from environment-specific symbolic and reinforcement learning agents, to current large foundation models acting as generalist players, and toward a future creator stage where agent both creates new game worlds and continually evolves within them. We trace the full lifecycle of a generalist game player along four interdependent pillars: Dataset, Model, Harness, and Benchmark. Every advance across these pillars can be read as an attempt to break one of five fundamental trade-offs that currently bound the whole system. Building on this end-to-end view, we chart a five-level roadmap, progressing from single-game mastery to the ultimate creator stage in which the agent simultaneously creates and evolves within theoretical game multiverse. Taken together, our work offers a unified lens onto a rapidly shifting field,and a principled path toward the omnipotent generalist agent capable of seamlessly mastering any challenge within the multiverse of games, thereby paving the way for AGI.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 150 of its 201 references that are in the map. First posted on arXiv 11 May 2026.
Builds on
150 of 201 references.
References · 150
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2022Training language models to follow instructions with human feedbackcited by 15,971
- 2017Proximal Policy Optimization Algorithmscited by 15,215
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
- 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
- 2024GPT-4o System Cardcited by 4,588
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
- 2023Qwen Technical Reportcited by 3,344
- 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
- 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,838
- 2025Qwen3-VL Technical Reportcited by 2,793
- 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
38 more
- 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,191
- 2023Efficient Streaming Language Models with Attention Sinkscited by 2,186
- 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,653
- 2022Fast Inference from Transformers via Speculative Decodingcited by 1,653
- 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
- 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,103
- 2020Mastering Atari with Discrete World Modelscited by 1,007
- 2023Open X-Embodiment: Robotic Learning Datasets and RT-X Modelscited by 862
- 2018Recurrent World Models Facilitate Policy Evolutioncited by 785
- 2024Genie: Generative Interactive Environmentscited by 747
- 2022BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learningcited by 523
- 2019Model-Based Reinforcement Learning for Ataricited by 502
- 2022MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledgecited by 478
- 2023Cognitive Architectures for Language Agentscited by 429
- 2022Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videoscited by 345
- 2022Transformers are Sample-Efficient World Modelscited by 328
- 2024Diffusion for World Modeling: Visual Details Matter in Ataricited by 300
- 2024Diffusion Models Are Real-Time Game Enginescited by 296
- 2017StarCraft II: A New Challenge for Reinforcement Learningcited by 274
- 2019MineRL: A Large-Scale Dataset of Minecraft Demonstrationscited by 215
- 2025UI-TARS-2 Technical Report: Advancing GUI Agent with Multi-Turn Reinforcement Learningcited by 215
- 2021Benchmarking the Spectrum of Agent Capabilitiescited by 200
- 2020The NetHack Learning Environmentcited by 198
- 2025Matrix-game 2.0: An open-source real-time and streaming interactive world modelcited by 186
- 2015Action-Conditional Video Prediction using Deep Networks in Atari Gamescited by 172
- 2025WorldPlay: Towards Long-Term Geometric Consistency for Real-Time Interactive World Modelingcited by 170
- 2019Interactive Fiction Games: A Colossal Adventurecited by 168
- 2025Ctrl-World: A Controllable Generative World Model for Robot Manipulationcited by 167
- 2023Language Agents with Reinforcement Learning for Strategic Play in the Werewolf Gamecited by 152
- 2024BALROG: Benchmarking Agentic LLM and VLM Reasoning On Gamescited by 146
- 2025GameFactory: Creating New Games with Generative Interactive Videoscited by 130
- 2023SmartPlay: A Benchmark for LLMs as Intelligent Agentscited by 125
- 2019Learning to Speak and Act in a Fantasy Text Adventure Gamecited by 122
- 2024Cradle: Empowering Foundation Agents Towards General Computer Controlcited by 117
- 2026Matrix-Game 3.0: Real-Time and Streaming Interactive World Model with Long-Horizon Memorycited by 100
- 2023STEVE-1: A Generative Model for Text-to-Behavior in Minecraftcited by 97
- 2025MineWorld: a Real-Time and Open-Source Interactive World Model on Minecraftcited by 97
- 2024GameBench: Evaluating Strategic Reasoning Abilities of LLM Agentscited by 95
50 most cited of 150 All 150 (JSON)
Built on by
Citing papers per year: 2026: 3.
3 papers.
Similar, not cited
Close in wording, no citation either way
- 2025Assessing Adaptive World Models in Machines with Novel Games
- 2026GameBoyWorlds: A Testbed for Self-Improvement in Embodied Video Games
- 2024A Survey on Large Language Model-Based Game Agents
- 2026Player-1: Learning to Play from Synthetic Experience
- 2022Multi-Game Decision Transformers
- 2025Robust Agents in Open-Ended Worlds
- 2026AI for Games in the Foundation Model Era
- 2024Massively Multiagent Minigames for Training Generalist Agents