mlsanityData to 7 Oct 2026

GSWorld: Closed-Loop Photo-Realistic Simulation Suite for Robotic Manipulation

arXiv 2510.20813 ↗First posted 23 Oct 2025cs.RO cs.AI cs.CVcited by 16

Not on the map. Why?

Abstract (arXiv)

This paper presents GSWorld, a robust, photo-realistic simulator for robotics manipulation that combines 3D Gaussian Splatting with physics engines. Our framework advocates "closing the loop" of developing manipulation policies with reproducible evaluation of policies learned from real-robot data and sim2real policy training without using real robots. To enable photo-realistic rendering of diverse scenes, we propose a new asset format, which we term GSDF (Gaussian Scene Description File), that infuses Gaussian-on-Mesh representation with robot URDF and other objects. With a streamlined reconstruction pipeline, we curate a database of GSDF that contains 3 robot embodiments for single-arm and bimanual manipulation, as well as more than 40 objects. Combining GSDF with physics engines, we demonstrate several immediate interesting applications: (1) learning zero-shot sim2real pixel-to-action manipulation policy with photo-realistic rendering, (2) automated high-quality DAgger data collection for adapting policies to deployment environments, (3) reproducible benchmarking of real-robot manipulation policies in simulation, (4) simulation data collection by virtual teleoperation, and (5) zero-shot sim2real visual reinforcement learning. Website: https://3dgsworld.github.io/.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Builds on 47 of its 64 references that are in the map. First posted on arXiv 23 Oct 2025.

Builds on

47 of 64 references.

References · 47

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
  3. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  4. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
  5. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,297
  6. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,490
  7. 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,483
  8. 2024Octo: An Open-Source Generalist Robot Policycited by 1,427
  9. 20242D Gaussian Splatting for Geometrically Accurate Radiance Fieldscited by 1,061
  10. 2024RDT-1B: a Diffusion Foundation Model for Bimanual Manipulationcited by 670
  11. 2019RLBench: The Robot Learning Benchmark & Learning Environmentcited by 669
  12. 2020SAPIEN: A SimulAted Part-based Interactive ENvironmentcited by 584
35 more
  1. 2024Evaluating Real-World Robot Manipulation Policies in Simulationcited by 484
  2. 2025RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulationcited by 477
  3. 2024RoboCasa: Large-Scale Simulation of Everyday Tasks for Generalist Robotscited by 391
  4. 2024Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robotscited by 373
  5. 2024ManiSkill3: GPU Parallelized Robotics Simulation and Rendering for Generalizable Embodied AIcited by 216
  6. 2024OmniRe: Omni Urban Scene Reconstructioncited by 159
  7. 2024Open-TeleVision: Teleoperation with Immersive Active Visual Feedbackcited by 137
  8. 2024Reconciling Reality through Simulation: A Real-to-Sim-to-Real Approach for Robust Manipulationcited by 135
  9. 2025RoboTwin: Dual-Arm Robot Benchmark with Generative Digital Twinscited by 117
  10. 2025EgoVLA: Learning Vision-Language-Action Models from Egocentric Human Videoscited by 116
  11. 2024ManiGaussian: Dynamic Gaussian Splatting for Multi-task Robotic Manipulationcited by 106
  12. 2024URDFormer: A Pipeline for Constructing Articulated Simulation Environments from Real-World Imagescited by 106
  13. 2022Robot Learning on the Job: Human-in-the-Loop Autonomy and Learning During Deploymentcited by 101
  14. 2024Automated Creation of Digital Cousins for Robust Policy Learningcited by 88
  15. 2025PhysTwin: Physics-Informed Reconstruction and Simulation of Deformable Objects from Videoscited by 78
  16. 2024Neural Implicit Representation for Building Digital Twins of Unknown Articulated Objectscited by 67
  17. 2024Real2Code: Reconstruct Articulated Objects via Code Generationcited by 64
  18. 2024SplatSim: Zero-Shot Sim2Real Transfer of RGB Manipulation Policies Using Gaussian Splattingcited by 63
  19. 2025MuJoCo Playgroundcited by 57
  20. 2023Diffusion Reward: Learning Rewards via Conditional Video Diffusioncited by 56
  21. 2025Amodal3R: Amodal 3D Reconstruction from Occluded 2D Imagescited by 52
  22. 2024RoboGSim: A Real2Sim2Real Robotic Gaussian Splatting Simulatorcited by 50
  23. 2025RoboVerse: Towards a Unified Platform, Dataset and Benchmark for Scalable and Generalizable Robot Learningcited by 48
  24. 2025Novel Demonstration Generation with Gaussian Splatting Enables Robust One-Shot Manipulationcited by 45
  25. 2024Robo-GS: A Physics Consistent Spatial-Temporal Model for Robotic Arm with Hybrid Representationcited by 44
  26. 2024GraspSplats: Efficient Manipulation with 3D Feature Splattingcited by 41
  27. 2025Re3Sim: Generating High-Fidelity Simulation Data via 3D-Photorealistic Real-to-Sim for Robotic Manipulationcited by 38
  28. 2024Diffusion Meets DAgger: Supercharging Eye-in-hand Imitation Learningcited by 34
  29. 2025Scalable Real2Sim: Physics-Aware Asset Generation Via Robotic Pick-and-Place Setupscited by 29
  30. 2025Digital Twin Catalog: A Large-Scale Photorealistic 3D Object Digital Twin Datasetcited by 29
  31. 2024Robots Pre-train Robots: Manipulation-Centric Robotic Representation from Large-Scale Robot Datasetscited by 23
  32. 2024Differentiable Robot Renderingcited by 20
  33. 2020Amodal 3D Reconstruction for Robotic Manipulation via Stability and Connectivitycited by 15
  34. 2024RoboDuet: Learning a Cooperative Policy for Whole-body Legged Loco-Manipulationcited by 6
  35. 2024LiveScene: Language Embedding Interactive Radiance Fields for Physical Scene Rendering and Controlcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 14.

16 papers.

Sort:

By area:

  1. 2025PolaRiS: Scalable Real-to-Sim Evaluations for Generalist Robot Policiescited by 29
  2. 2025π_textttRL: Online RL Fine-tuning for Flow-based Vision-Language-Action Modelscited by 16
  3. 2026Kinema4D: Kinematic 4D World Modeling for Spatiotemporal Embodied Simulationcited by 9
  4. 2026TwinRL: Digital Twin-Driven Reinforcement Learning for Real-World Robotic Manipulationcited by 5
  5. 2026Cross-Hand Latent Representation for Vision-Language-Action Modelscited by 5
  6. 2026Reconstruction by Generation: 3D Multi-Object Scene Reconstruction from Sparse Observationscited by 5
  7. 2026SAIL: Test-Time Scaling for In-Context Imitation Learning with VLMcited by 2
  8. 2026DREAM: Deployment-Time Demonstration Generation via Real-to-Sim for Scalable Policy Adaptationcited by 1
  9. 2026Mirage2Matter: A Physically Grounded Gaussian World Model from Videocited by 0
  10. 2026LEGS: Fine-Tuning Teleop-Free VLAs for Humanoid Loco-manipulation in an Embodied Gaussian Splatting Worldcited by 0
  11. 2026EgoPhys: Learning Generalizable Physics Models of Deformable Objects from Egocentric Videocited by 0
  12. 2026DeepInsight II: One Trace from Benchmark to Robotcited by 0
4 more
  1. 2026Where Success Breaks: Failure-Boundary Learning for Robust Vision-Language-Action Modelscited by 0
  2. 2026The Neverwhere Visual Parkour Benchmark Suitecited by 0
  3. 2026φ-RIE: From Photorealistic Reconstruction to Interactive Environmentscited by 0
  4. 2026PointCast: One World Model for Rigid, Articulated, and Deformable Object Manipulationcited by 0