mlsanityData to 14 Oct 2026

Towards Generalist Game Players: An Investigation of Foundation Models in the Game Multiverse

arXiv 2605.09965 ↗First posted 11 May 2026cs.CVcited by 3

Not on the map. Why?

Abstract (arXiv)

The real world unfolds along a single set of physics laws, yet human intelligence demonstrates a remarkable capacity to generalize experiences from this singular physical existence into a multiverse of games, each governed by entirely different rules, aesthetics, physics, and objectives. This omni-reality adaptability is a hallmark of general intelligence. As Artificial Intelligence progresses towards Artificial General Intelligence, the multiverse of games has evolved from mere entertainment into the ultimate ground for training and evaluating AGI. The pursuit of this generality has unfolded across four eras: from environment-specific symbolic and reinforcement learning agents, to current large foundation models acting as generalist players, and toward a future creator stage where agent both creates new game worlds and continually evolves within them. We trace the full lifecycle of a generalist game player along four interdependent pillars: Dataset, Model, Harness, and Benchmark. Every advance across these pillars can be read as an attempt to break one of five fundamental trade-offs that currently bound the whole system. Building on this end-to-end view, we chart a five-level roadmap, progressing from single-game mastery to the ultimate creator stage in which the agent simultaneously creates and evolves within theoretical game multiverse. Taken together, our work offers a unified lens onto a rapidly shifting field,and a principled path toward the omnipotent generalist agent capable of seamlessly mastering any challenge within the multiverse of games, thereby paving the way for AGI.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 150 of its 201 references that are in the map. First posted on arXiv 11 May 2026.

Builds on

150 of 201 references.

References · 150

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2017Proximal Policy Optimization Algorithmscited by 15,215
  4. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  5. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  6. 2024GPT-4o System Cardcited by 4,588
  7. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
  8. 2023Qwen Technical Reportcited by 3,344
  9. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
  10. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,838
  11. 2025Qwen3-VL Technical Reportcited by 2,793
  12. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
38 more
  1. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,191
  2. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,186
  3. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,653
  4. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,653
  5. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
  6. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,103
  7. 2020Mastering Atari with Discrete World Modelscited by 1,007
  8. 2023Open X-Embodiment: Robotic Learning Datasets and RT-X Modelscited by 862
  9. 2018Recurrent World Models Facilitate Policy Evolutioncited by 785
  10. 2024Genie: Generative Interactive Environmentscited by 747
  11. 2022BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learningcited by 523
  12. 2019Model-Based Reinforcement Learning for Ataricited by 502
  13. 2022MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledgecited by 478
  14. 2023Cognitive Architectures for Language Agentscited by 429
  15. 2022Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videoscited by 345
  16. 2022Transformers are Sample-Efficient World Modelscited by 328
  17. 2024Diffusion for World Modeling: Visual Details Matter in Ataricited by 300
  18. 2024Diffusion Models Are Real-Time Game Enginescited by 296
  19. 2017StarCraft II: A New Challenge for Reinforcement Learningcited by 274
  20. 2019MineRL: A Large-Scale Dataset of Minecraft Demonstrationscited by 215
  21. 2025UI-TARS-2 Technical Report: Advancing GUI Agent with Multi-Turn Reinforcement Learningcited by 215
  22. 2021Benchmarking the Spectrum of Agent Capabilitiescited by 200
  23. 2020The NetHack Learning Environmentcited by 198
  24. 2025Matrix-game 2.0: An open-source real-time and streaming interactive world modelcited by 186
  25. 2015Action-Conditional Video Prediction using Deep Networks in Atari Gamescited by 172
  26. 2025WorldPlay: Towards Long-Term Geometric Consistency for Real-Time Interactive World Modelingcited by 170
  27. 2019Interactive Fiction Games: A Colossal Adventurecited by 168
  28. 2025Ctrl-World: A Controllable Generative World Model for Robot Manipulationcited by 167
  29. 2023Language Agents with Reinforcement Learning for Strategic Play in the Werewolf Gamecited by 152
  30. 2024BALROG: Benchmarking Agentic LLM and VLM Reasoning On Gamescited by 146
  31. 2025GameFactory: Creating New Games with Generative Interactive Videoscited by 130
  32. 2023SmartPlay: A Benchmark for LLMs as Intelligent Agentscited by 125
  33. 2019Learning to Speak and Act in a Fantasy Text Adventure Gamecited by 122
  34. 2024Cradle: Empowering Foundation Agents Towards General Computer Controlcited by 117
  35. 2026Matrix-Game 3.0: Real-Time and Streaming Interactive World Model with Long-Horizon Memorycited by 100
  36. 2023STEVE-1: A Generative Model for Text-to-Behavior in Minecraftcited by 97
  37. 2025MineWorld: a Real-Time and Open-Source Interactive World Model on Minecraftcited by 97
  38. 2024GameBench: Evaluating Strategic Reasoning Abilities of LLM Agentscited by 95

50 most cited of 150 All 150 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025Assessing Adaptive World Models in Machines with Novel Games
  2. 2026GameBoyWorlds: A Testbed for Self-Improvement in Embodied Video Games
  3. 2024A Survey on Large Language Model-Based Game Agents
  4. 2026Player-1: Learning to Play from Synthetic Experience
  5. 2022Multi-Game Decision Transformers
  6. 2025Robust Agents in Open-Ended Worlds
  7. 2026AI for Games in the Foundation Model Era
  8. 2024Massively Multiagent Minigames for Training Generalist Agents