mlsanityData to 7 Oct 2026

SimWorld Studio: Automatic Environment Generation with Evolving Coding Agent for Embodied Agent Learning

arXiv 2605.09423 ↗First posted 10 May 2026cs.AIcited by 13

On the map: LLM agents and tool use → Multimodal and embodied agents

Abstract (arXiv)

LLM/VLM-based digital agents have advanced rapidly thanks to scalable sandboxes for coding, web navigation, and computer use, which provide rich interactive training grounds. In contrast, embodied agents still lack abundant, diverse, and automatically generated 3D environments for interactive learning. Existing embodied simulators rely on manually crafted scenes or procedural templates, while recent LLM-based 3D generation systems mainly produce static scenes rather than deployable environments with verifiable tasks and standard learning interfaces. We introduce SimWorld Studio, an open-source platform built on Unreal Engine 5 for generating evolving embodied learning environments. At its core is SimCoder, a tool/skill-augmented coding agent that writes and executes engine-level code to construct physically grounded 3D worlds from language/image instructions. SimCoder self-evolves by using verifier feedback (e.g., compilation errors, physics checks, VLM critiques) to revise environments and autonomously add reusable tools and skills to its library. Generated worlds are exported as Gym-style environments for embodied agent learning. SimWorld Studio further enables co-evolution between environment generation and embodied learning: agent performance feedback guides SimCoder to generate adaptive curricula near the learner's capability frontier, so that environments become increasingly challenging as the embodied agent improves. Three case studies on embodied navigation show that self-evolution improves generation reliability, generated environments substantially improve embodied agent performance that generalizes to unseen benchmarks, and co-evolution yields an 18-point success-rate gain over fixed-environment learning and a 40-point gain over an untrained agent.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 80 of its 101 references that are in the map. First posted on arXiv 10 May 2026.

Builds on

80 of 101 references.

Closest ancestors · 6

Same direction, newest first

  1. 2026SAGE: Scalable Agentic 3D Scene Generation for Embodied AI10 Feb
  2. 2026SceneSmith: Agentic Generation of Simulation-Ready Indoor Scenes9 Feb
  3. 2025VULCAN: Tool-Augmented Multi Agents for Iterative 3D Object Arrangement26 Dec
  4. 2024EmbodiedCity: A Benchmark Platform for Embodied Agent in Real-world City Environment12 Oct
  5. 2024LEGENT: Open Platform for Embodied Agents28 Apr
  6. 2024SceneCraft: An LLM Agent for Synthesizing 3D Scene as Blender Code2 Mar

Other references · 74

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  4. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  5. 2021Program Synthesis with Large Language Modelscited by 3,509
  6. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  7. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  8. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
  9. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
  10. 2017CARLA: An Open Urban Driving Simulatorcited by 2,431
  11. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,906
  12. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
38 more
  1. 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,761
  2. 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,628
  3. 2023AgentBench: Evaluating LLMs as Agentscited by 1,211
  4. 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,143
  5. 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,134
  6. 2024OpenHands: An Open Platform for AI Software Developers as Generalist Agentscited by 956
  7. 2025Cosmos World Foundation Model Platform for Physical AIcited by 833
  8. 2023ExpeL: LLM Agents Are Experiential Learnerscited by 814
  9. 2024Genie: Generative Interactive Environmentscited by 706
  10. 2023Eureka: Human-Level Reward Design via Coding Large Language Modelscited by 542
  11. 2024Executable Code Actions Elicit Better LLM Agentscited by 537
  12. 2024BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructionscited by 527
  13. 2018On Evaluation of Embodied Navigation Agentscited by 526
  14. 2022MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledgecited by 469
  15. 2023Learning Interactive Real-World Simulatorscited by 440
  16. 2024Automated Design of Agentic Systemscited by 360
  17. 2025Agentic Context Engineering: Evolving Contexts for Self-Improving Language Modelscited by 349
  18. 2025Darwin Godel Machine: Open-Ended Evolution of Self-Improving Agentscited by 294
  19. 2020Emergent Complexity and Zero-shot Transfer via Unsupervised Environment Designcited by 260
  20. 2020ThreeDWorld: A Platform for Interactive Multi-Modal Physical Simulationcited by 249
  21. 2023Text2Room: Extracting Textured 3D Meshes from 2D Text-to-Image Modelscited by 245
  22. 2024WebRL: Training LLM Web Agents via Self-Evolving Online Curriculum Reinforcement Learningcited by 243
  23. 2020ObjectNav Revisited: On Evaluation of Embodied Agents Navigating to Objectscited by 240
  24. 2023LucidDreamer: Domain-free Generation of 3D Gaussian Splatting Scenescited by 228
  25. 2025R-Zero: Self-Evolving Reasoning LLM from Zero Datacited by 210
  26. 2023RoboGen: Towards Unleashing Infinite Data for Automated Robot Learning via Generative Simulationcited by 206
  27. 2024WonderWorld: Interactive 3D Scene Generation from a Single Imagecited by 197
  28. 2024SWE-bench Multimodal: Do AI Systems Generalize to Visual Software Domains?cited by 196
  29. 2021ATISS: Autoregressive Transformers for Indoor Scene Synthesiscited by 195
  30. 2021iGibson 2.0: Object-Centric Simulation for Robot Learning of Everyday Household Taskscited by 191
  31. 2020Prioritized Level Replaycited by 190
  32. 2025A Comprehensive Survey of Self-Evolving AI Agents: A New Paradigm Bridging Foundation Models and Lifelong Agentic Systemscited by 177
  33. 2022Evolving Curricula with Regret-Based Environment Designcited by 166
  34. 2025Multi-SWE-bench: A Multilingual Benchmark for Issue Resolvingcited by 165
  35. 2023DiffuScene: Denoising Diffusion Models for Generative Indoor Scene Synthesiscited by 159
  36. 2025A Survey of Self-Evolving Agents: What, When, How, and Where to Evolve on the Path to Artificial Super Intelligencecited by 155
  37. 2023Text2Reward: Reward Shaping with Language Models for Reinforcement Learningcited by 153
  38. 2023Infinite Photorealistic Worlds using Procedural Generationcited by 142

50 most cited of 74 All 74 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 13.

13 papers.

Sort:

By area:

Similar, not cited

Close in wording, no citation either way

  1. 2026EmbodiedGen V2: An Agentic, Simulation-Ready 3D World Engine for Embodied AI
  2. 2026VirtualEnv: A Platform for Embodied AI Research
  3. 2026Video2World: Benchmarking Coding Agents for Interactive World Modeling from Embodied Videos
  4. 2025SimWorld: An Open-ended Realistic Simulator for Autonomous Agents in Physical and Social Worlds
  5. 20263D Generation for Embodied AI and Robotic Simulation: A Survey
  6. 2024EnvGen: Generating and Adapting Environments via LLMs for Training Embodied Agents
  7. 2025LatticeWorld: A Multimodal Large Language Model-Empowered Framework for Interactive Complex World Generation
  8. 2026Awomo-SimDataEngine: Agentic Simulation-ReadyWorld Generation