Game-TARS: Pretrained Foundation Models for Scalable Generalist Multimodal Game Agents
On the map: LLM agents and tool use → Reinforcement learning for LLM agents
Abstract (arXiv)
We present Game-TARS, a generalist game agent trained with a unified, scalable action space anchored to human-aligned native keyboard-mouse inputs. Unlike API- or GUI-based approaches, this paradigm enables large-scale continual pre-training across heterogeneous domains, including OS, web, and simulation games. Game-TARS is pre-trained on over 500B tokens with diverse trajectories and multimodal data. Key techniques include a decaying continual loss to reduce causal confusion and an efficient Sparse-Thinking strategy that balances reasoning depth and inference cost. Experiments show that Game-TARS achieves about 2 times the success rate over the previous sota model on open-world Minecraft tasks, is close to the generality of fresh humans in unseen web 3d games, and outperforms GPT-5, Gemini-2.5-Pro, and Claude-4-Sonnet in FPS benchmarks. Scaling results on training-time and test-time confirm that the unified action space sustains improvements when scaled to cross-game and multimodal data. Our results demonstrate that simple, scalable action representations combined with large-scale pre-training provide a promising path toward generalist agents with broad computer-use abilities.
Key facts. Cited by 23 papers in the map (data through 7 Oct 2026). Placed in Reinforcement learning for LLM agents, one of 16 directions in LLM agents and tool use. Builds on 59 of its 76 references that are in the map. First posted on arXiv 27 Oct 2025.
Builds on
59 of 76 references.
Closest ancestors · 9
Same direction, newest first
- 2025OpenHA: A Series of Open-Source Hierarchical Agentic Models in Minecraft13 Sep
- 2025Acting Less is Reasoning More! Teaching Model to Act Efficiently21 Apr
- 2025ReTool: Reinforcement Learning for Strategic Tool Use in LLMs15 Apr
- 2024Optimus-1: Hybrid Multimodal Memory Empowered Agents Excel in Long-Horizon Tasks7 Aug
- 2024OmniJARVIS: Unified Vision-Language-Action Tokenization Enables Open-World Instruction Following Agents27 Jun
- 2024Fine-Tuning Large Vision-Language Models as Decision-Making Agents via Reinforcement Learning16 May
- 2023JARVIS-1: Open-World Multi-task Agents with Memory-Augmented Multimodal Language Models10 Nov
- 2023Voyager: An Open-Ended Embodied Agent with Large Language Models25 May
- 2022MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledge17 Jun
Other references · 50
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
- 2025Qwen2.5-VL Technical Reportcited by 5,418
- 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
- 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
- 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
- 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
- 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
- 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,653
- 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,628
- 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,490
38 more
- 2023Mind2Web: Towards a Generalist Agent for the Webcited by 1,210
- 2023Mastering Diverse Domains through World Modelscited by 1,193
- 2025GR00T N1: An Open Foundation Model for Generalist Humanoid Robotscited by 1,167
- 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,134
- 2023Open X-Embodiment: Robotic Learning Datasets and RT-X Modelscited by 849
- 2022A Generalist Agentcited by 802
- 2024Genie: Generative Interactive Environmentscited by 706
- 2023CogAgent: A Visual Language Model for GUI Agentscited by 692
- 2025BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agentscited by 616
- 2025UI-TARS: Pioneering Automated GUI Interaction with Native Agentscited by 584
- 2022VIMA: General Robot Manipulation with Multimodal Promptscited by 403
- 2024OS-ATLAS: A Foundation Action Model for Generalist GUI Agentscited by 370
- 2025Seed1.5-VL Technical Reportcited by 367
- 2022Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videoscited by 338
- 2025R1-Searcher: Incentivizing the Search Capability in LLMs via Reinforcement Learningcited by 322
- 2023Minigrid & Miniworld: Modular & Customizable Reinforcement Learning Environments for Goal-Oriented Taskscited by 320
- 2019Causal Confusion in Imitation Learningcited by 286
- 2024Aguvis: Unified Pure Vision Agents for Autonomous GUI Interactioncited by 271
- 2025ScreenSpot-Pro: GUI Grounding for Professional High-Resolution Computer Usecited by 246
- 2025GUI-R1 : A Generalist R1-Style Vision-Language Action Model For GUI Agentscited by 240
- 2025UI-TARS-2 Technical Report: Advancing GUI Agent with Multi-Turn Reinforcement Learningcited by 205
- 2024RT-H: Action Hierarchies Using Languagecited by 204
- 2025Seed1.5-Thinking: Advancing Superb Reasoning Models with Reinforcement Learningcited by 159
- 2025OpenCUA: Open Foundations for Computer-Use Agentscited by 151
- 2024Exploring Large Language Model based Intelligent Agents: Definitions, Methods, and Prospectscited by 130
- 2024Cradle: Empowering Foundation Agents Towards General Computer Controlcited by 114
- 2025GR-3 Technical Reportcited by 110
- 2023ProAgent: Building Proactive Cooperative Agents with Large Language Modelscited by 97
- 2023STEVE-1: A Generative Model for Text-to-Behavior in Minecraftcited by 95
- 2024RAT: Retrieval Augmented Thoughts Elicit Context-Aware Reasoning in Long-Horizon Generationcited by 75
- 2025ARPO:End-to-End Policy Optimization for GUI Agents with Experience Replaycited by 75
- 2024Scaling Instructable Agents Across Many Simulated Worldscited by 67
- 2023Auto MC-Reward: Automated Dense Reward Design with Large Language Models for Minecraftcited by 60
- 2018ViZDoom Competitions: Playing Doom from Pixelscited by 50
- 2023Skill Reinforcement Learning and Planning for Open-World Long-Horizon Taskscited by 35
- 2025VideoGameBench: Can Vision-Language Models complete popular video games?cited by 29
- 2025JARVIS-VLA: Post-Training Large-Scale Vision Language Models to Play Visual Games with Keyboards and Mousecited by 27
- 2025VeOmni: Scaling Any Modality Model Training with Model-Centric Distributed Recipe Zoocited by 20
Built on by
Citing papers per year: 2025: 3; 2026: 20.
23 papers.
Sort:
By area:
- 2025SIMA 2: A Generalist Embodied Agent for Virtual Worldscited by 26
- 2026Beyond Trajectory-Level Attribution: Graph-Based Credit Assignment for Agentic Reinforcement Learningcited by 23
- 2026OS-Symphony: A Holistic Framework for Robust and Generalist Computer-Using Agentcited by 19
- 2026Show-Harness: Just a VLM Agent Can Play Robotscited by 17
- 2026Scaling Behavior Cloning Improves Causal Reasoning: An Open Model for Real-Time Video Game Playingcited by 10
- 2026AI Gamestore: Scalable, Open-Ended Evaluation of Machine General Intelligence with Human Gamescited by 10
- 2026Agentic World Modeling: Foundations, Capabilities, Laws, and Beyondcited by 10
- 2026GameVerse: Can Vision-Language Models Learn from Video-based Reflection?cited by 7
- 2026OmniGameArena: A Unified UE5 Benchmark for VLM Game Agents with Improvement Dynamicscited by 4
- 2026MapTab: A Diagnostic Benchmark for Long-Horizon Multi-Criteria Multimodal Reasoning on Heterogeneous Topological Graphscited by 3
- 2025IPR-1: Interactive Physical Reasonercited by 2
- 2026MAIN-VLA: Modeling Abstraction of Intention and eNvironment for Vision-Language-Action Modelscited by 2
11 more
- 2026Odysseus: Scaling VLMs to 100+ Turn Decision-Making in Games via Reinforcement Learningcited by 2
- 2026Towards Generalist Game Players: An Investigation of Foundation Models in the Game Multiversecited by 2
- 2026AI for Games in the Foundation Model Eracited by 2
- 2026Scaling, Benchmarking, and Reasoning of Vision-Language Agents for Mobile GUI Navigationcited by 1
- 2026ActSWM: Action-Sensitive World Models for Long-Horizon Planning in Open-World Gamescited by 1
- 2025Training One Model to Master Cross-Level Agentic Actions via Reinforcement Learningcited by 0
- 2026Stratagem: Learning Transferable Reasoning via Trajectory-Modulated Game Self-Playcited by 0
- 2026GROW: Aligning GRPO with State-Action Modeling for Open-World VLM Agentscited by 0
- 2026GameWAM: A World Action Model for Video Gamescited by 0
- 2026GameHorizon Suite: Multi-Horizon Data and Evaluation in Gameplaycited by 0
- 2026VISTA: A Visual Harness for Reasoning in an Interactive Worldcited by 0
- 2026VISTA: A Visual Harness for Reasoning in an Interactive World1 Oct 2026
- 2026GameHorizon Suite: Multi-Horizon Data and Evaluation in Gameplay21 Sep 2026
- 2026AI for Games in the Foundation Model Era15 Sep 2026
- 2026Show-Harness: Just a VLM Agent Can Play Robots9 Sep 2026
- 2026GameWAM: A World Action Model for Video Games25 Aug 2026
- 2026ActSWM: Action-Sensitive World Models for Long-Horizon Planning in Open-World Games29 Jul 2026
- 2026OmniGameArena: A Unified UE5 Benchmark for VLM Game Agents with Improvement Dynamics8 Jun 2026
- 2026Scaling, Benchmarking, and Reasoning of Vision-Language Agents for Mobile GUI Navigation26 May 2026
- 2026Beyond Trajectory-Level Attribution: Graph-Based Credit Assignment for Agentic Reinforcement Learning26 May 2026
- 2026GROW: Aligning GRPO with State-Action Modeling for Open-World VLM Agents18 May 2026
- 2026Towards Generalist Game Players: An Investigation of Foundation Models in the Game Multiverse11 May 2026
- 2026Odysseus: Scaling VLMs to 100+ Turn Decision-Making in Games via Reinforcement Learning1 May 2026
11 more
- 2026Agentic World Modeling: Foundations, Capabilities, Laws, and Beyond24 Apr 2026
- 2026Stratagem: Learning Transferable Reasoning via Trajectory-Modulated Game Self-Play20 Apr 2026
- 2026GameVerse: Can Vision-Language Models Learn from Video-based Reflection?1 Mar 2026
- 2026MapTab: A Diagnostic Benchmark for Long-Horizon Multi-Criteria Multimodal Reasoning on Heterogeneous Topological Graphs20 Feb 2026
- 2026AI Gamestore: Scalable, Open-Ended Evaluation of Machine General Intelligence with Human Games19 Feb 2026
- 2026MAIN-VLA: Modeling Abstraction of Intention and eNvironment for Vision-Language-Action Models2 Feb 2026
- 2026OS-Symphony: A Holistic Framework for Robust and Generalist Computer-Using Agent12 Jan 2026
- 2026Scaling Behavior Cloning Improves Causal Reasoning: An Open Model for Real-Time Video Game Playing8 Jan 2026
- 2025Training One Model to Master Cross-Level Agentic Actions via Reinforcement Learning10 Dec 2025
- 2025SIMA 2: A Generalist Embodied Agent for Virtual Worlds4 Dec 2025
- 2025IPR-1: Interactive Physical Reasoner19 Nov 2025
- 2026Scaling Behavior Cloning Improves Causal Reasoning: An Open Model for Real-Time Video Game Playingcited by 10
- 2026MapTab: A Diagnostic Benchmark for Long-Horizon Multi-Criteria Multimodal Reasoning on Heterogeneous Topological Graphscited by 3
- 2026Scaling, Benchmarking, and Reasoning of Vision-Language Agents for Mobile GUI Navigationcited by 1
- 2026Stratagem: Learning Transferable Reasoning via Trajectory-Modulated Game Self-Playcited by 0
- 2026GROW: Aligning GRPO with State-Action Modeling for Open-World VLM Agentscited by 0
- 2026VISTA: A Visual Harness for Reasoning in an Interactive Worldcited by 0
- 2026VISTA: A Visual Harness for Reasoning in an Interactive World1 Oct 2026
- 2026Scaling, Benchmarking, and Reasoning of Vision-Language Agents for Mobile GUI Navigation26 May 2026
- 2026GROW: Aligning GRPO with State-Action Modeling for Open-World VLM Agents18 May 2026
- 2026Stratagem: Learning Transferable Reasoning via Trajectory-Modulated Game Self-Play20 Apr 2026
- 2026MapTab: A Diagnostic Benchmark for Long-Horizon Multi-Criteria Multimodal Reasoning on Heterogeneous Topological Graphs20 Feb 2026
- 2026Scaling Behavior Cloning Improves Causal Reasoning: An Open Model for Real-Time Video Game Playing8 Jan 2026
- 2026Agentic World Modeling: Foundations, Capabilities, Laws, and Beyondcited by 10
- 2025IPR-1: Interactive Physical Reasonercited by 2
- 2026AI for Games in the Foundation Model Eracited by 2
- 2026ActSWM: Action-Sensitive World Models for Long-Horizon Planning in Open-World Gamescited by 1
- 2026GameWAM: A World Action Model for Video Gamescited by 0
- 2026AI for Games in the Foundation Model Era15 Sep 2026
- 2026GameWAM: A World Action Model for Video Games25 Aug 2026
- 2026ActSWM: Action-Sensitive World Models for Long-Horizon Planning in Open-World Games29 Jul 2026
- 2026Agentic World Modeling: Foundations, Capabilities, Laws, and Beyond24 Apr 2026
- 2025IPR-1: Interactive Physical Reasoner19 Nov 2025
- 2025SIMA 2: A Generalist Embodied Agent for Virtual Worldscited by 26
- 2026Beyond Trajectory-Level Attribution: Graph-Based Credit Assignment for Agentic Reinforcement Learningcited by 23
- 2026OmniGameArena: A Unified UE5 Benchmark for VLM Game Agents with Improvement Dynamicscited by 4
- 2025Training One Model to Master Cross-Level Agentic Actions via Reinforcement Learningcited by 0
- 2026OmniGameArena: A Unified UE5 Benchmark for VLM Game Agents with Improvement Dynamics8 Jun 2026
- 2026Beyond Trajectory-Level Attribution: Graph-Based Credit Assignment for Agentic Reinforcement Learning26 May 2026
- 2025Training One Model to Master Cross-Level Agentic Actions via Reinforcement Learning10 Dec 2025
- 2025SIMA 2: A Generalist Embodied Agent for Virtual Worlds4 Dec 2025
- 2026OS-Symphony: A Holistic Framework for Robust and Generalist Computer-Using Agentcited by 19
- 2026Show-Harness: Just a VLM Agent Can Play Robotscited by 17
- 2026GameVerse: Can Vision-Language Models Learn from Video-based Reflection?cited by 7
- 2026Odysseus: Scaling VLMs to 100+ Turn Decision-Making in Games via Reinforcement Learningcited by 2
- 2026Show-Harness: Just a VLM Agent Can Play Robots9 Sep 2026
- 2026Odysseus: Scaling VLMs to 100+ Turn Decision-Making in Games via Reinforcement Learning1 May 2026
- 2026GameVerse: Can Vision-Language Models Learn from Video-based Reflection?1 Mar 2026
- 2026OS-Symphony: A Holistic Framework for Robust and Generalist Computer-Using Agent12 Jan 2026
Similar, not cited
Close in wording, no citation either way
- 2026GameWorld: Towards Standardized and Verifiable Evaluation of Multimodal Game Agents
- 2024TeamCraft: A Benchmark for Multi-Modal Multi-Agent Systems in Minecraft
- 2026NitroGen: An Open Foundation Model for Generalist Gaming Agents
- 2025Training Agents Inside of Scalable World Models
- 2026MineExplorer: Evaluating Open-World Exploration of MLLM Agents in Minecraft
- 2025VistaWise: Building Cost-Effective Agent with Cross-Modal Knowledge Graph for Minecraft
- 2025Collaborating Action by Action: A Multi-agent LLM Framework for Embodied Reasoning
- 2025Orak: A Foundational Benchmark for Training and Evaluating LLM Agents on Diverse Video Games