# RL-GSBridge: 3D Gaussian Splatting Based Real2Sim2Real Method for Robotic Manipulation Learning

arXiv: https://arxiv.org/abs/2409.20291 · HTML: https://mlsanity.com/paper/2409.20291/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Sep 2024.

## Built on by (most cited first, 22 of 22)

- [VR-Robo: A Real-to-Sim-to-Real Framework for Visual Robot Navigation and Locomotion](https://mlsanity.com/paper/2502.01536/) (2025)
- [Towards Physically Executable 3D Gaussian for Embodied Navigation](https://mlsanity.com/paper/2510.21307/) (2025)
- [CCL-LGS: Contrastive Codebook Learning for 3D Language Gaussian Splatting](https://mlsanity.com/paper/2505.20469/) (2025)
- [EmbodiedSplat: Personalized Real-to-Sim-to-Real Navigation with Gaussian Splats from a Mobile Device](https://mlsanity.com/paper/2509.17430/) (2025)
- [HAIF-GS: Hierarchical and Induced Flow-Guided Gaussian Splatting for Dynamic Scene](https://mlsanity.com/paper/2506.09518/) (2025)
- [Beyond Imitation: Reinforcement Learning-Based Sim-Real Co-Training for VLA Models](https://mlsanity.com/paper/2602.12628/) (2026)
- [Phys2Real: Fusing VLM Priors with Interactive Online Adaptation for Uncertainty-Aware Sim-to-Real Manipulation](https://mlsanity.com/paper/2510.11689/) (2025)
- [Bootstrapping Imitation Learning for Long-horizon Manipulation via Hierarchical Data Collection Space](https://mlsanity.com/paper/2505.17389/) (2025)
- [RLGS: Reinforcement Learning-Based Adaptive Hyperparameter Tuning for Gaussian Splatting](https://mlsanity.com/paper/2508.04078/) (2025)
- [FAR-Dex: Few-shot Data Augmentation and Adaptive Residual Policy Refinement for Dexterous Manipulation](https://mlsanity.com/paper/2603.10451/) (2026)
- [VLK: Learning Humanoid Loco-Manipulation from Synthetic Interactions in Reconstructed Scenes](https://mlsanity.com/paper/2606.30645/) (2026)
- [PRISM: Projection-based Reward Integration for Scene-Aware Real-to-Sim-to-Real Transfer with Few Demonstrations](https://mlsanity.com/paper/2504.20520/) (2025)
- [AOMGen: Photoreal, Physics-Consistent Demonstration Generation for Articulated Object Manipulation](https://mlsanity.com/paper/2512.18396/) (2025)
- [ViserDex: Visual Sim-to-Real for Robust Dexterous In-hand Reorientation](https://mlsanity.com/paper/2604.11138/) (2026)
- [UAVTwin: Neural Digital Twins for UAVs using Gaussian Splatting](https://mlsanity.com/paper/2504.02158/) (2025)
- [REAP: Reinforcement-Learning End-to-End Autonomous Parking with Gaussian Splatting Simulator for Real2Sim2Real Transfer](https://mlsanity.com/paper/2605.08713/) (2026)
- [Splat2Real: Novel-view Scaling for Physical AI with 3D Gaussian Splatting](https://mlsanity.com/paper/2603.10638/) (2026)
- [ManiSplat: Manipulation Trajectory Synthesis from Monocular Video via Decoupled 3D Gaussian Splatting](https://mlsanity.com/paper/2606.10645/) (2026)
- [ReVeal: A Reconstruction-Aware Real-to-Sim Framework for VLA Policy Evaluation](https://mlsanity.com/paper/2609.23910/) (2026)
- [LiTe-GS: Oracle-Efficient Next Best View Selection for 3D Gaussian Splatting](https://mlsanity.com/paper/2609.30393/) (2026)
- [F4R: Failure-Driven Recognition, Reconstruction, Refinement, and Redeployment for Continual Robot Self-Improvement](https://mlsanity.com/paper/2609.35575/) (2026)
- [Splat2Act: From Captured Scenes to Personalized VLA Manipulation](https://mlsanity.com/paper/2610.13945/) (2026)

## Abstract (arXiv)

Sim-to-Real refers to the process of transferring policies learned in simulation to the real world, which is crucial for achieving practical robotics applications. However, recent Sim2real methods either rely on a large amount of augmented data or large learning models, which is inefficient for specific tasks. In recent years, with the emergence of radiance field reconstruction methods, especially 3D Gaussian splatting, it has become possible to construct realistic real-world scenes. To this end, we propose RL-GSBridge, a novel real-to-sim-to-real framework which incorporates 3D Gaussian Splatting into the conventional RL simulation pipeline, enabling zero-shot sim-to-real transfer for vision-based deep reinforcement learning. We introduce a mesh-based 3D GS method with soft binding constraints, enhancing the rendering quality of mesh models. Then utilizing a GS editing approach to synchronize the rendering with the physics simulator, RL-GSBridge could reflect the visual interactions of the physical robot accurately. Through a series of sim-to-real experiments, including grasping and pick-and-place tasks, we demonstrate that RL-GSBridge maintains a satisfactory success rate in real-world task completion during sim-to-real transfer. Furthermore, a series of rendering metrics and visualization results indicate that our proposed mesh-based 3D GS reduces artifacts in unstructured objects, demonstrating more realistic rendering performance.