mlsanityData to 7 Oct 2026

SR-Platform: An Agentic Pipeline for Natural Language-Driven Robot Simulation Environment Synthesis

arXiv 2605.14700 ↗First posted 14 May 2026cs.ROcited by 0

On the map: LLM agents and tool use → Multimodal and embodied agents

Abstract (arXiv)

Generating robot simulation environments remains a major bottleneck in simulation-based robot learning. Constructing a training-ready MuJoCo scene typically requires expertise in 3D asset modeling, MJCF specification, spatial layout, collision avoidance, and robot-model integration. We present SR-Platform, a production-deployed agentic system that converts free-form natural language descriptions into executable, physically valid MuJoCo environments. SR-Platform decomposes scene synthesis into four stages: an LLM-based orchestrator that converts user intent into a structured scene plan; an asset forge that retrieves cached assets or generates new 3D geometry through LLM-to-CadQuery synthesis; a layout architect that assigns object poses and verifies industrial constraints; and a bridge layer that assembles the final MJCF scene and merges the selected robot model. The system is deployed as a nine-service Docker stack with WebSocket progress streaming, MinIO-backed mesh storage, Qdrant-based semantic asset retrieval, Redis job state, and InfluxDB telemetry. Using 30 days of production telemetry covering 611 successful LLM calls, SR-Platform generates five-object scenes with a median end-to-end latency of approximately 50 s, while cache-accelerated scenes complete in approximately 30-40 s. The asset forge shows an 11.3% first-attempt retry rate with automatic recovery, and cached asset retrieval removes per-object LLM calls for previously generated object types. These results show that agentic scene synthesis can reduce the manual effort required to create diverse robot training environments, enabling users to produce executable MuJoCo scenes from plain English prompts in under one minute.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 15 of its 23 references that are in the map. First posted on arXiv 14 May 2026.

Builds on

15 of 23 references.

References · 15

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  4. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
  5. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
  6. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
  7. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
  8. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,401
  9. 2022Code as Policies: Language Model Programs for Embodied Controlcited by 1,172
  10. 2023Eureka: Human-Level Reward Design via Coding Large Language Modelscited by 542
  11. 2020Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Surveycited by 388
  12. 2020ThreeDWorld: A Platform for Interactive Multi-Modal Physical Simulationcited by 249
3 more
  1. 2023Orbit: A Unified Simulation Framework for Interactive Robot Learning Environmentscited by 239
  2. 2023RoboGen: Towards Unleashing Infinite Data for Automated Robot Learning via Generative Simulationcited by 206
  3. 2018Sim-to-Real Reinforcement Learning for Deformable Object Manipulationcited by 124

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026SceneSmith: Agentic Generation of Simulation-Ready Indoor Scenes
  2. 2026Awomo-SimDataEngine: Agentic Simulation-ReadyWorld Generation
  3. 2026Agentic Real2Sim: Physics-based World Modeling with Vision-Language Agents
  4. 2026SimWorlds: A Multi-Agent System for Dynamic 3D Scene Creation
  5. 2026Agentic RSR: Real-to-Sim-to-Real through Scene Reconstruction and Execution-Grounded Robot Policies
  6. 2026Genie Sim 3.0 : A High-Fidelity Comprehensive Simulation Platform for Humanoid Robot
  7. 2024AutoRT: Embodied Foundation Models for Large Scale Orchestration of Robotic Agents
  8. 2024GRS: Generating Robotic Simulation Tasks from Real-World Images