mlsanityData to 7 Oct 2026

SIMA 2: A Generalist Embodied Agent for Virtual Worlds

arXiv 2512.04797 ↗First posted 4 Dec 2025cs.AI cs.ROcited by 26

On the map: LLM agents and tool use → Reinforcement learning for LLM agents

Abstract (arXiv)

We introduce SIMA 2, a generalist embodied agent that understands and acts in a wide variety of 3D virtual worlds. Built upon a Gemini foundation model, SIMA 2 represents a significant step toward active, goal-directed interaction within an embodied environment. Unlike prior work (e.g., SIMA 1) limited to simple language commands, SIMA 2 acts as an interactive partner, capable of reasoning about high-level goals, conversing with the user, and handling complex instructions given through language and images. Across a diverse portfolio of games, SIMA 2 substantially closes the gap with human performance and demonstrates robust generalization to previously unseen environments, all while retaining the base model's core reasoning capabilities. Furthermore, we demonstrate a capacity for open-ended self-improvement: by leveraging Gemini to generate tasks and provide rewards, SIMA 2 can autonomously learn new skills from scratch in a new environment. This work validates a path toward creating versatile and continuously learning agents for both virtual and, eventually, physical worlds.

Key facts. Cited by 26 papers in the map (data through 7 Oct 2026). Placed in Reinforcement learning for LLM agents, one of 16 directions in LLM agents and tool use. Builds on 88 of its 123 references that are in the map. First posted on arXiv 4 Dec 2025.

Builds on

88 of 123 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025Lumine: An Open Recipe for Building Generalist Agents in 3D Open Worlds12 Nov
  2. 2025Game-TARS: Pretrained Foundation Models for Scalable Generalist Multimodal Game Agents27 Oct
  3. 2024BALROG: Benchmarking Agentic LLM and VLM Reasoning On Games20 Nov
  4. 2023JARVIS-1: Open-World Multi-task Agents with Memory-Augmented Multimodal Language Models10 Nov
  5. 2023Voyager: An Open-Ended Embodied Agent with Large Language Models25 May
  6. 2022MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledge17 Jun

Other references · 82

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  3. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
  4. 2023Qwen Technical Reportcited by 3,273
  5. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
  6. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
  7. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
  8. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
  9. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,297
  10. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
  11. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
  12. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 1,906
38 more
  1. 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,653
  2. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
  3. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,221
  4. 2019Habitat: A Platform for Embodied AI Researchcited by 1,163
  5. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,084
  6. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,013
  7. 2017AI2-THOR: An Interactive 3D Environment for Visual AIcited by 850
  8. 2023Open X-Embodiment: Robotic Learning Datasets and RT-X Modelscited by 849
  9. 2015End-to-End Training of Deep Visuomotor Policiescited by 847
  10. 2022A Generalist Agentcited by 802
  11. 2018Recurrent World Models Facilitate Policy Evolutioncited by 761
  12. 2024Genie: Generative Interactive Environmentscited by 706
  13. 2022R3M: A Universal Visual Representation for Robot Manipulationcited by 678
  14. 2023An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuningcited by 622
  15. 2023GAIA-1: A Generative World Model for Autonomous Drivingcited by 618
  16. 2021CLIPort: What and Where Pathways for Robotic Manipulationcited by 617
  17. 2023Eureka: Human-Level Reward Design via Coding Large Language Modelscited by 542
  18. 2022Phenaki: Variable Length Video Generation From Open Domain Textual Descriptioncited by 475
  19. 2025CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Modelscited by 440
  20. 2022ProcTHOR: Large-Scale Embodied AI Using Procedural Generationcited by 369
  21. 2020Planning to Explore via Self-Supervised World Modelscited by 365
  22. 2016Target-driven Visual Navigation in Indoor Scenes using Deep Reinforcement Learningcited by 351
  23. 2022Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videoscited by 338
  24. 2017Adversarial Attacks on Neural Network Policiescited by 314
  25. 2023Language to Rewards for Robotic Skill Synthesiscited by 307
  26. 2024Diffusion Models Are Real-Time Game Enginescited by 282
  27. 2025Reinforcement Learning with Verifiable Rewards Implicitly Incentivizes Correct Reasoning in Base LLMscited by 276
  28. 2018Visual Reinforcement Learning with Imagined Goalscited by 248
  29. 2025EmbodiedBench: Comprehensive Benchmarking Multi-modal Large Language Models for Vision-Driven Embodied Agentscited by 213
  30. 2022Multi-Game Decision Transformerscited by 212
  31. 2019MineRL: A Large-Scale Dataset of Minecraft Demonstrationscited by 210
  32. 2017Cognitive Mapping and Planning for Visual Navigationcited by 209
  33. 2015Supersizing Self-supervision: Learning to Grasp from 50K Tries and 700 Robot Hourscited by 208
  34. 2023Guiding Pretraining in Reinforcement Learning with Large Language Modelscited by 185
  35. 2019Paired Open-Ended Trailblazer (POET): Endlessly Generating Increasingly Complex and Diverse Learning Environments and Their Solutionscited by 182
  36. 2023LIV: Language-Image Representations and Rewards for Robotic Controlcited by 180
  37. 2016ViZDoom: A Doom-based AI Research Platform for Visual Reinforcement Learningcited by 178
  38. 2025GAIA-2: A Controllable Multi-View Generative World Model for Autonomous Drivingcited by 178

50 most cited of 82 All 82 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 26.

26 papers.

Sort:

By area:

  1. 2026Graph-based Agent Memory: Taxonomy, Techniques, and Applicationscited by 22
  2. 2026GameDevBench: Evaluating Agentic Capabilities Through Game Developmentcited by 21
  3. 2026OpenGame: Open Agentic Coding for Gamescited by 12
  4. 2026Scaling Behavior Cloning Improves Causal Reasoning: An Open Model for Real-Time Video Game Playingcited by 10
  5. 2026Agentic Environment Engineering for Large Language Models: A Survey of Environment Modeling, Synthesis, Evaluation, and Applicationcited by 8
  6. 2026SPADE: Self-Play in Adaptive Synthetic Executable Environmentscited by 6
  7. 2026The Last AI Built by Humans: Toward Genuine Recursive Self-Improvementcited by 5
  8. 2026GameplayQA: A Benchmarking Framework for Decision-Dense POV-Synced Multi-Video Understanding of 3D Virtual Agentscited by 3
  9. 2026PhysMem: Scaling Test-Time Memory for Embodied Physical Reasoningcited by 2
  10. 2026Can Vision-Language Models Solve the Shell Game?cited by 2
  11. 2026Odysseus: Scaling VLMs to 100+ Turn Decision-Making in Games via Reinforcement Learningcited by 2
  12. 2026Towards Generalist Game Players: An Investigation of Foundation Models in the Game Multiversecited by 2
14 more
  1. 2026Critique of Agent Modelcited by 2
  2. 2026AI for Games in the Foundation Model Eracited by 2
  3. 2026Deep Research of Deep Research: From Transformer to Agent, From AI to AI for Sciencecited by 1
  4. 2026Scalable Environments Drive Generalizable Agentscited by 1
  5. 2026Multi-agent Framework for Time-Sensitive Complementary Collaboration in Minecraftcited by 1
  6. 2026Latent Goal Prediction from Language for Model-Based Planningcited by 1
  7. 2026SyncPlan: Long-Horizon LLM Coordination with Explicit Synchronization and Adaptive Correctioncited by 1
  8. 2026A Vision for Multisensory Intelligence: Sensing, Science, and Synergycited by 0
  9. 2026Learning Bilevel Policies over Symbolic World Models for Long-Horizon Planningcited by 0
  10. 2026VLMs for Videogame Data Annotationcited by 0
  11. 2026Contrastive World Modelscited by 0
  12. 2026GameHorizon Suite: Multi-Horizon Data and Evaluation in Gameplaycited by 0
  13. 2026SkillVine: Agent Skill Evolution via Branching Explorationcited by 0
  14. 2026Distilling Agentic Systems: A Roadmap across Models, Artifacts, and Harnessescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026VirtualEnv: A Platform for Embodied AI Research
  2. 2026SimWorld Studio: Automatic Environment Generation with Evolving Coding Agent for Embodied Agent Learning
  3. 2023See and Think: Embodied Agent in Virtual Environment
  4. 2026NitroGen: An Open Foundation Model for Generalist Gaming Agents
  5. 2026GameWorld: Towards Standardized and Verifiable Evaluation of Multimodal Game Agents
  6. 2025CrafterDojo: A Suite of Foundation Models for Building Open-Ended Embodied Agents in Crafter
  7. 2026GameBoyWorlds: A Testbed for Self-Improvement in Embodied Video Games
  8. 2024ADAM: An Embodied Causal Agent in Open-World Environments