SR-Platform: An Agentic Pipeline for Natural Language-Driven Robot Simulation Environment Synthesis
On the map: LLM agents and tool use → Multimodal and embodied agents
Abstract (arXiv)
Generating robot simulation environments remains a major bottleneck in simulation-based robot learning. Constructing a training-ready MuJoCo scene typically requires expertise in 3D asset modeling, MJCF specification, spatial layout, collision avoidance, and robot-model integration. We present SR-Platform, a production-deployed agentic system that converts free-form natural language descriptions into executable, physically valid MuJoCo environments. SR-Platform decomposes scene synthesis into four stages: an LLM-based orchestrator that converts user intent into a structured scene plan; an asset forge that retrieves cached assets or generates new 3D geometry through LLM-to-CadQuery synthesis; a layout architect that assigns object poses and verifies industrial constraints; and a bridge layer that assembles the final MJCF scene and merges the selected robot model. The system is deployed as a nine-service Docker stack with WebSocket progress streaming, MinIO-backed mesh storage, Qdrant-based semantic asset retrieval, Redis job state, and InfluxDB telemetry. Using 30 days of production telemetry covering 611 successful LLM calls, SR-Platform generates five-object scenes with a median end-to-end latency of approximately 50 s, while cache-accelerated scenes complete in approximately 30-40 s. The asset forge shows an 11.3% first-attempt retry rate with automatic recovery, and cached asset retrieval removes per-object LLM calls for previously generated object types. These results show that agentic scene synthesis can reduce the manual effort required to create diverse robot training environments, enabling users to produce executable MuJoCo scenes from plain English prompts in under one minute.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 15 of its 23 references that are in the map. First posted on arXiv 14 May 2026.
Builds on
15 of 23 references.
References · 15
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
- 2021Evaluating Large Language Models Trained on Codecited by 8,053
- 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
- 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
- 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
- 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
- 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,401
- 2022Code as Policies: Language Model Programs for Embodied Controlcited by 1,172
- 2023Eureka: Human-Level Reward Design via Coding Large Language Modelscited by 542
- 2020Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Surveycited by 388
- 2020ThreeDWorld: A Platform for Interactive Multi-Modal Physical Simulationcited by 249
3 more
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026SceneSmith: Agentic Generation of Simulation-Ready Indoor Scenes
- 2026Awomo-SimDataEngine: Agentic Simulation-ReadyWorld Generation
- 2026Agentic Real2Sim: Physics-based World Modeling with Vision-Language Agents
- 2026SimWorlds: A Multi-Agent System for Dynamic 3D Scene Creation
- 2026Agentic RSR: Real-to-Sim-to-Real through Scene Reconstruction and Execution-Grounded Robot Policies
- 2026Genie Sim 3.0 : A High-Fidelity Comprehensive Simulation Platform for Humanoid Robot
- 2024AutoRT: Embodied Foundation Models for Large Scale Orchestration of Robotic Agents
- 2024GRS: Generating Robotic Simulation Tasks from Real-World Images