SimWorld Studio: Automatic Environment Generation with Evolving Coding Agent for Embodied Agent Learning
On the map: LLM agents and tool use → Multimodal and embodied agents
Abstract (arXiv)
LLM/VLM-based digital agents have advanced rapidly thanks to scalable sandboxes for coding, web navigation, and computer use, which provide rich interactive training grounds. In contrast, embodied agents still lack abundant, diverse, and automatically generated 3D environments for interactive learning. Existing embodied simulators rely on manually crafted scenes or procedural templates, while recent LLM-based 3D generation systems mainly produce static scenes rather than deployable environments with verifiable tasks and standard learning interfaces. We introduce SimWorld Studio, an open-source platform built on Unreal Engine 5 for generating evolving embodied learning environments. At its core is SimCoder, a tool/skill-augmented coding agent that writes and executes engine-level code to construct physically grounded 3D worlds from language/image instructions. SimCoder self-evolves by using verifier feedback (e.g., compilation errors, physics checks, VLM critiques) to revise environments and autonomously add reusable tools and skills to its library. Generated worlds are exported as Gym-style environments for embodied agent learning. SimWorld Studio further enables co-evolution between environment generation and embodied learning: agent performance feedback guides SimCoder to generate adaptive curricula near the learner's capability frontier, so that environments become increasingly challenging as the embodied agent improves. Three case studies on embodied navigation show that self-evolution improves generation reliability, generated environments substantially improve embodied agent performance that generalizes to unseen benchmarks, and co-evolution yields an 18-point success-rate gain over fixed-environment learning and a 40-point gain over an untrained agent.
Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 80 of its 101 references that are in the map. First posted on arXiv 10 May 2026.
Builds on
80 of 101 references.
Closest ancestors · 6
Same direction, newest first
- 2026SAGE: Scalable Agentic 3D Scene Generation for Embodied AI10 Feb
- 2026SceneSmith: Agentic Generation of Simulation-Ready Indoor Scenes9 Feb
- 2025VULCAN: Tool-Augmented Multi Agents for Iterative 3D Object Arrangement26 Dec
- 2024EmbodiedCity: A Benchmark Platform for Embodied Agent in Real-world City Environment12 Oct
- 2024LEGENT: Open Platform for Embodied Agents28 Apr
- 2024SceneCraft: An LLM Agent for Synthesizing 3D Scene as Blender Code2 Mar
Other references · 74
- 2017Proximal Policy Optimization Algorithmscited by 14,441
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
- 2021Evaluating Large Language Models Trained on Codecited by 8,053
- 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
- 2021Program Synthesis with Large Language Modelscited by 3,509
- 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
- 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
- 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
- 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
- 2017CARLA: An Open Urban Driving Simulatorcited by 2,431
- 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,906
- 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
38 more
- 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,761
- 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,628
- 2023AgentBench: Evaluating LLMs as Agentscited by 1,211
- 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,143
- 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,134
- 2024OpenHands: An Open Platform for AI Software Developers as Generalist Agentscited by 956
- 2025Cosmos World Foundation Model Platform for Physical AIcited by 833
- 2023ExpeL: LLM Agents Are Experiential Learnerscited by 814
- 2024Genie: Generative Interactive Environmentscited by 706
- 2023Eureka: Human-Level Reward Design via Coding Large Language Modelscited by 542
- 2024Executable Code Actions Elicit Better LLM Agentscited by 537
- 2024BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructionscited by 527
- 2018On Evaluation of Embodied Navigation Agentscited by 526
- 2022MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledgecited by 469
- 2023Learning Interactive Real-World Simulatorscited by 440
- 2024Automated Design of Agentic Systemscited by 360
- 2025Agentic Context Engineering: Evolving Contexts for Self-Improving Language Modelscited by 349
- 2025Darwin Godel Machine: Open-Ended Evolution of Self-Improving Agentscited by 294
- 2020Emergent Complexity and Zero-shot Transfer via Unsupervised Environment Designcited by 260
- 2020ThreeDWorld: A Platform for Interactive Multi-Modal Physical Simulationcited by 249
- 2023Text2Room: Extracting Textured 3D Meshes from 2D Text-to-Image Modelscited by 245
- 2024WebRL: Training LLM Web Agents via Self-Evolving Online Curriculum Reinforcement Learningcited by 243
- 2020ObjectNav Revisited: On Evaluation of Embodied Agents Navigating to Objectscited by 240
- 2023LucidDreamer: Domain-free Generation of 3D Gaussian Splatting Scenescited by 228
- 2025R-Zero: Self-Evolving Reasoning LLM from Zero Datacited by 210
- 2023RoboGen: Towards Unleashing Infinite Data for Automated Robot Learning via Generative Simulationcited by 206
- 2024WonderWorld: Interactive 3D Scene Generation from a Single Imagecited by 197
- 2024SWE-bench Multimodal: Do AI Systems Generalize to Visual Software Domains?cited by 196
- 2021ATISS: Autoregressive Transformers for Indoor Scene Synthesiscited by 195
- 2021iGibson 2.0: Object-Centric Simulation for Robot Learning of Everyday Household Taskscited by 191
- 2020Prioritized Level Replaycited by 190
- 2025A Comprehensive Survey of Self-Evolving AI Agents: A New Paradigm Bridging Foundation Models and Lifelong Agentic Systemscited by 177
- 2022Evolving Curricula with Regret-Based Environment Designcited by 166
- 2025Multi-SWE-bench: A Multilingual Benchmark for Issue Resolvingcited by 165
- 2023DiffuScene: Denoising Diffusion Models for Generative Indoor Scene Synthesiscited by 159
- 2025A Survey of Self-Evolving Agents: What, When, How, and Where to Evolve on the Path to Artificial Super Intelligencecited by 155
- 2023Text2Reward: Reward Shaping with Language Models for Reinforcement Learningcited by 153
- 2023Infinite Photorealistic Worlds using Procedural Generationcited by 142
50 most cited of 74 All 74 (JSON)
Built on by
Citing papers per year: 2026: 13.
13 papers.
Sort:
By area:
- 2026Agentic Real2Sim: Physics-based World Modeling with Vision-Language Agentscited by 6
- 2026WorldClaw: Agentic 3D Open-World Generation at Scalecited by 6
- 2026The Last AI Built by Humans: Toward Genuine Recursive Self-Improvementcited by 5
- 2026EmbodiedSWE: Coding Agents for Long Horizon Dexterous Roboticscited by 2
- 2026Co-Evolution in Agentic Systems: Toward Self-Directed Evolution Beyond Human Designcited by 1
- 2026DeliveryGym: An RL Environment for Long-Horizon Embodied Agent Planning with Adaptive Curriculumcited by 1
- 2026Agentic Visual Generation: From Generative Models to Agentic Controlcited by 0
- 2026DiagGen: Agentic Generation of Deformable Assets with Sim-based Diagnostics for Robotic Simulationcited by 0
- 2026WorldAgent: Verification-Guided Agentic Physical World Constructioncited by 0
- 2026LEGO-Anything: Coding Agents for 3D Scene Reconstructioncited by 0
- 2026Code4Scene: Benchmarking Coding Agents for Constructing and Editing 3D Scenescited by 0
- 2026Agentic RSR: Real-to-Sim-to-Real through Scene Reconstruction and Execution-Grounded Robot Policiescited by 0
1 more
- 2026AgentGarten: Code Worlds for Evolving Agentscited by 0
- 2026AgentGarten: Code Worlds for Evolving Agents8 Oct 2026
- 2026Agentic RSR: Real-to-Sim-to-Real through Scene Reconstruction and Execution-Grounded Robot Policies7 Oct 2026
- 2026Code4Scene: Benchmarking Coding Agents for Constructing and Editing 3D Scenes29 Sep 2026
- 2026LEGO-Anything: Coding Agents for 3D Scene Reconstruction28 Sep 2026
- 2026WorldAgent: Verification-Guided Agentic Physical World Construction27 Sep 2026
- 2026EmbodiedSWE: Coding Agents for Long Horizon Dexterous Robotics23 Sep 2026
- 2026DiagGen: Agentic Generation of Deformable Assets with Sim-based Diagnostics for Robotic Simulation19 Sep 2026
- 2026DeliveryGym: An RL Environment for Long-Horizon Embodied Agent Planning with Adaptive Curriculum17 Sep 2026
- 2026The Last AI Built by Humans: Toward Genuine Recursive Self-Improvement10 Sep 2026
- 2026Agentic Visual Generation: From Generative Models to Agentic Control6 Sep 2026
- 2026Co-Evolution in Agentic Systems: Toward Self-Directed Evolution Beyond Human Design10 Aug 2026
- 2026WorldClaw: Agentic 3D Open-World Generation at Scale5 Aug 2026
1 more
- 2026WorldClaw: Agentic 3D Open-World Generation at Scalecited by 6
- 2026EmbodiedSWE: Coding Agents for Long Horizon Dexterous Roboticscited by 2
- 2026Co-Evolution in Agentic Systems: Toward Self-Directed Evolution Beyond Human Designcited by 1
- 2026DeliveryGym: An RL Environment for Long-Horizon Embodied Agent Planning with Adaptive Curriculumcited by 1
- 2026Agentic Visual Generation: From Generative Models to Agentic Controlcited by 0
- 2026DiagGen: Agentic Generation of Deformable Assets with Sim-based Diagnostics for Robotic Simulationcited by 0
- 2026WorldAgent: Verification-Guided Agentic Physical World Constructioncited by 0
- 2026Code4Scene: Benchmarking Coding Agents for Constructing and Editing 3D Scenescited by 0
- 2026Agentic RSR: Real-to-Sim-to-Real through Scene Reconstruction and Execution-Grounded Robot Policiescited by 0
- 2026Agentic RSR: Real-to-Sim-to-Real through Scene Reconstruction and Execution-Grounded Robot Policies7 Oct 2026
- 2026Code4Scene: Benchmarking Coding Agents for Constructing and Editing 3D Scenes29 Sep 2026
- 2026WorldAgent: Verification-Guided Agentic Physical World Construction27 Sep 2026
- 2026EmbodiedSWE: Coding Agents for Long Horizon Dexterous Robotics23 Sep 2026
- 2026DiagGen: Agentic Generation of Deformable Assets with Sim-based Diagnostics for Robotic Simulation19 Sep 2026
- 2026DeliveryGym: An RL Environment for Long-Horizon Embodied Agent Planning with Adaptive Curriculum17 Sep 2026
- 2026Agentic Visual Generation: From Generative Models to Agentic Control6 Sep 2026
- 2026Co-Evolution in Agentic Systems: Toward Self-Directed Evolution Beyond Human Design10 Aug 2026
- 2026WorldClaw: Agentic 3D Open-World Generation at Scale5 Aug 2026
- 2026LEGO-Anything: Coding Agents for 3D Scene Reconstructioncited by 0
- 2026AgentGarten: Code Worlds for Evolving Agentscited by 0
- 2026AgentGarten: Code Worlds for Evolving Agents8 Oct 2026
- 2026LEGO-Anything: Coding Agents for 3D Scene Reconstruction28 Sep 2026
Similar, not cited
Close in wording, no citation either way
- 2026EmbodiedGen V2: An Agentic, Simulation-Ready 3D World Engine for Embodied AI
- 2026VirtualEnv: A Platform for Embodied AI Research
- 2026Video2World: Benchmarking Coding Agents for Interactive World Modeling from Embodied Videos
- 2025SimWorld: An Open-ended Realistic Simulator for Autonomous Agents in Physical and Social Worlds
- 20263D Generation for Embodied AI and Robotic Simulation: A Survey
- 2024EnvGen: Generating and Adapting Environments via LLMs for Training Embodied Agents
- 2025LatticeWorld: A Multimodal Large Language Model-Empowered Framework for Interactive Complex World Generation
- 2026Awomo-SimDataEngine: Agentic Simulation-ReadyWorld Generation