# SimLauncher: Launching Sample-Efficient Real-world Robotic Reinforcement Learning via Simulation Pre-training arXiv: https://arxiv.org/abs/2507.04452 ยท HTML: https://mlsanity.com/paper/2507.04452/ ## Key facts - Cited by 5 papers in the map (data through 14 Oct 2026). - Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods. - Builds on 50 of its 57 references that are in the map. - First posted on arXiv 6 Jul 2025. ## Placed in - Deep reinforcement learning methods / [RL for robot locomotion and manipulation](https://mlsanity.com/deeprl/rl-for-robot-locomotion-and-manipulation/) ## Builds on (most cited first, 30 of 50) - [SAM 2: Segment Anything in Images and Videos](https://mlsanity.com/paper/2408.00714/) (2024) - [Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learning](https://mlsanity.com/paper/2108.10470/) (2021) - [Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrations](https://mlsanity.com/paper/1709.10087/) (2017) - [AWAC: Accelerating Online Reinforcement Learning with Offline Datasets](https://mlsanity.com/paper/2006.09359/) (2020) - [BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learning](https://mlsanity.com/paper/2202.02005/) (2022) - [Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Survey](https://mlsanity.com/paper/2009.13303/) (2020) - [RoboCasa: Large-Scale Simulation of Everyday Tasks for Generalist Robots](https://mlsanity.com/paper/2406.02523/) (2024) - [Efficient Online Reinforcement Learning with Offline Data](https://mlsanity.com/paper/2302.02948/) (2023) - [MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrations](https://mlsanity.com/paper/2310.17596/) (2023) - [RoboGen: Towards Unleashing Infinite Data for Automated Robot Learning via Generative Simulation](https://mlsanity.com/paper/2311.01455/) (2023) - [ConRFT: A Reinforced Fine-tuning Method for VLA Models via Consistency Policy](https://mlsanity.com/paper/2502.05450/) (2025) - [Reconciling Reality through Simulation: A Real-to-Sim-to-Real Approach for Robust Manipulation](https://mlsanity.com/paper/2403.03949/) (2024) - [SERL: A Software Suite for Sample-Efficient Robotic Reinforcement Learning](https://mlsanity.com/paper/2401.16013/) (2024) - [ALOHA Unleashed: A Simple Recipe for Robot Dexterity](https://mlsanity.com/paper/2410.13126/) (2024) - [THE COLOSSEUM: A Benchmark for Evaluating Generalization for Robotic Manipulation](https://mlsanity.com/paper/2402.08191/) (2024) - [GenSim: Generating Robotic Simulation Tasks via Large Language Models](https://mlsanity.com/paper/2310.01361/) (2023) - [Towards Human-Level Bimanual Dexterous Manipulation with Reinforcement Learning](https://mlsanity.com/paper/2206.08686/) (2022) - [In-Hand Object Rotation via Rapid Motor Adaptation](https://mlsanity.com/paper/2210.04887/) (2022) - [The Ingredients of Real-World Robotic Reinforcement Learning](https://mlsanity.com/paper/2004.12570/) (2020) - [DexMimicGen: Automated Data Generation for Bimanual Dexterous Manipulation via Imitation Learning](https://mlsanity.com/paper/2410.24185/) (2024) - [Improving Vision-Language-Action Model with Online Reinforcement Learning](https://mlsanity.com/paper/2501.16664/) (2025) - [Reset-Free Reinforcement Learning via Multi-Task Learning: Learning Dexterous Manipulation Behaviors without Human Intervention](https://mlsanity.com/paper/2104.11203/) (2021) - [Automated Creation of Digital Cousins for Robust Policy Learning](https://mlsanity.com/paper/2410.07408/) (2024) - [Manipulate-Anything: Automating Real-World Robots using Vision-Language Models](https://mlsanity.com/paper/2406.18915/) (2024) - [Decomposing the Generalization Gap in Imitation Learning for Visual Robotic Manipulation](https://mlsanity.com/paper/2307.03659/) (2023) - [MuJoCo Playground](https://mlsanity.com/paper/2502.08844/) (2025) - [Robo-ABC: Affordance Generalization Beyond Categories via Semantic Correspondence for Robot Manipulation](https://mlsanity.com/paper/2401.07487/) (2024) - [RoboTwin: Dual-Arm Robot Benchmark with Generative Digital Twins (early version)](https://mlsanity.com/paper/2409.02920/) (2024) - [SplatSim: Zero-Shot Sim2Real Transfer of RGB Manipulation Policies Using Gaussian Splatting](https://mlsanity.com/paper/2409.10161/) (2024) - [Imitation Bootstrapped Reinforcement Learning](https://mlsanity.com/paper/2311.02198/) (2023) ## Built on by (most cited first, 5 of 5) - [TwinRL: Digital Twin-Driven Reinforcement Learning for Real-World Robotic Manipulation](https://mlsanity.com/paper/2602.09023/) (2026) - [Accelerating Robotic Reinforcement Learning with Agent Guidance](https://mlsanity.com/paper/2602.11978/) (2026) - [Towards Batch-to-Streaming Deep Reinforcement Learning for Continuous Control](https://mlsanity.com/paper/2603.08588/) (2026) - [ROAD: Adaptive Data Mixing for Offline-to-Online Reinforcement Learning via Bi-Level Optimization](https://mlsanity.com/paper/2605.14497/) (2026) - [Hierarchical Residual Policy Learning for Real-World Mobile Manipulation with Sparse Human Guidance](https://mlsanity.com/paper/2610.20607/) (2026) ## Abstract (arXiv) Autonomous learning of dexterous, long-horizon robotic skills has been a longstanding pursuit of embodied AI. Recent advances in robotic reinforcement learning (RL) have demonstrated remarkable performance and robustness in real-world visuomotor control tasks. However, applying RL in the real world faces challenges such as low sample efficiency, slow exploration, and significant reliance on human intervention. In contrast, simulators offer a safe and efficient environment for extensive exploration and data collection, while the visual sim-to-real gap, often a limiting factor, can be mitigated using real-to-sim techniques. Building on these, we propose SimLauncher, a novel framework that combines the strengths of real-world RL and real-to-sim-to-real approaches to overcome these challenges. Specifically, we first pre-train a visuomotor policy in the digital twin simulation environment, which then benefits real-world RL in two ways: (1) bootstrapping target values using extensive simulated demonstrations and real-world demonstrations derived from pre-trained policy rollouts, and (2) Incorporating action proposals from the pre-trained policy for better exploration. We conduct comprehensive experiments across multi-stage, contact-rich, and dexterous hand manipulation tasks. Compared to prior real-world RL approaches, SimLauncher significantly improves sample efficiency and achieves near-perfect success rates. We hope this work serves as a proof of concept and inspires further research on leveraging large-scale simulation pre-training to benefit real-world robotic RL.