An Real-Sim-Real (RSR) Loop Framework for Generalizable Robotic Policy Transfer
Not on the map. Why?
Abstract (arXiv)
The sim-to-real gap remains a critical challenge in robotics, hindering the deployment of algorithms trained in simulation to real-world systems. We propose a flexible Real-to-Sim-to-Real (RSR) framework whose central contribution is an information-theoretic cost function that explicitly accounts for sim-to-real discrepancies. This cost balances two objectives, completing the task and steering the policy to collect real-world samples that are maximally informative for improving transfer. It can be integrated seamlessly into existing reinforcement learning algorithms (e.g., PPO, SAC) and ensures a balanced exploration of critical regions in the real domain. The framework treats differentiable simulation as optional: when a differentiable simulator is available, the collected informative data can also be used to tune simulator parameters. We implement the framework with the MuJoCo MJX platform and demonstrate its generality by evaluating on both manipulation tasks with a 6-DOF robotic arm and locomotion tasks on a legged robot. Empirical results show that our RSR loop yields more efficient data acquisition and substantially improves task performance in real-world that achieves a smoother sim-to-real transfer.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 18 of its 37 references that are in the map. First posted on arXiv 13 Mar 2025.
Builds on
18 of 37 references.
References · 18
- 2015Domain-Adversarial Training of Neural Networkscited by 3,044
- 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,498
- 2020Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Surveycited by 413
- 2016Domain Separation Networkscited by 320
- 2017Using Simulation and Domain Adaptation to Improve Efficiency of Deep Robotic Graspingcited by 144
- 2024Reconciling Reality through Simulation: A Real-to-Sim-to-Real Approach for Robust Manipulationcited by 144
- 2025ASAP: Aligning Simulation and Real-World Physics for Learning Agile Humanoid Whole-Body Skillscited by 130
- 2017A Tutorial on Kernel Density Estimation and Recent Advancescited by 101
- 2021gradSim: Differentiable simulation for system identification and visuomotor controlcited by 100
- 2019BayesSim: adaptive domain randomization via probabilistic inference for robotics simulatorscited by 85
- 2020NeuralSim: Augmenting Differentiable Simulators with Neural Networkscited by 82
- 2021Efficient Differentiable Simulation of Articulated Bodiescited by 39
6 more
- 2024A Review of Differentiable Simulatorscited by 28
- 2024Differentiable Robot Renderingcited by 21
- 2024Learning Quadrotor Control From Visual Features Using Differentiable Simulationcited by 19
- 2025Bridging the Sim-to-Real Gap for Athletic Loco-Manipulationcited by 13
- 2024Dynamics as Prompts: In-Context Learning for Sim-to-Real System Identificationscited by 10
- 2024Bridging the Resource Gap: Deploying Advanced Imitation Learning Models onto Affordable Embedded Platformscited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Sim-to-Real Transfer in Deep Reinforcement Learning for Bipedal Locomotion
- 2025Towards bridging the gap: Systematic sim-to-real transfer for diverse legged robots
- 2024LoopSR: Looping Sim-and-Real for Lifelong Policy Adaptation of Legged Robots
- 2026Support-Constrained RL Enables Real-World Policy Improvement without Real-World Experience
- 2025Robot Policy Evaluation for Sim-to-Real Transfer: A Benchmarking Perspective
- 2024Improving generalization of robot locomotion policies via Sharpness-Aware Reinforcement Learning
- 2026Abstract Sim2Real through Approximate Information States
- 2025Can Context Bridge the Reality Gap? Sim-to-Real Transfer of Context-Aware Policies