# Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Survey

arXiv: https://arxiv.org/abs/2009.13303 · HTML: https://mlsanity.com/paper/2009.13303/

## Key facts

- Cited by 413 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Sep 2020.

## Built on by (most cited first, 30 of 413)

- [Learning Interactive Real-World Simulators](https://mlsanity.com/paper/2310.06114/) (2023)
- [A Survey of Zero-shot Generalisation in Deep Reinforcement Learning](https://mlsanity.com/paper/2111.09794/) (2021)
- [SoundSpaces 2.0: A Simulation Platform for Visual-Acoustic Learning](https://mlsanity.com/paper/2206.08312/) (2022)
- [What Matters in Building Vision-Language-Action Models for Generalist Robots](https://mlsanity.com/paper/2412.14058/) (2024)
- [WorldGym: World Model as An Environment for Policy Evaluation](https://mlsanity.com/paper/2506.00613/) (2025)
- [Enforcing Hard Constraints with Soft Barriers: Safe Reinforcement Learning in Unknown Stochastic Environments](https://mlsanity.com/paper/2209.15090/) (2022)
- [Residual Off-Policy RL for Finetuning Behavior Cloning Policies](https://mlsanity.com/paper/2509.19301/) (2025)
- [FLaRe: Achieving Masterful and Adaptive Robot Policies with Large-Scale Reinforcement Learning Fine-Tuning](https://mlsanity.com/paper/2409.16578/) (2024)
- [Mind the Sim2Real Gap in User Simulation for Agentic Tasks](https://mlsanity.com/paper/2603.11245/) (2026)
- [Humanoid-Gym: Reinforcement Learning for Humanoid Robot with Zero-Shot Sim2Real Transfer](https://mlsanity.com/paper/2404.05695/) (2024)
- [RoboGSim: A Real2Sim2Real Robotic Gaussian Splatting Simulator](https://mlsanity.com/paper/2411.11839/) (2024)
- [GenSim2: Scaling Robot Data Generation with Multi-modal and Reasoning LLMs](https://mlsanity.com/paper/2410.03645/) (2024)
- [Double Pessimism is Provably Efficient for Distributionally Robust Offline Reinforcement Learning: Generic Algorithm and Robust Partial Coverage](https://mlsanity.com/paper/2305.09659/) (2023)
- [VLATest: Testing and Evaluating Vision-Language-Action Models for Robotic Manipulation](https://mlsanity.com/paper/2409.12894/) (2024)
- [BadRobot: Jailbreaking Embodied LLM Agents in the Physical World](https://mlsanity.com/paper/2407.20242/) (2024)
- [PoCo: Policy Composition from and for Heterogeneous Robot Learning](https://mlsanity.com/paper/2402.02511/) (2024)
- [Finetuning Offline World Models in the Real World](https://mlsanity.com/paper/2310.16029/) (2023)
- [Towards Generalist Robot Learning from Internet Video: A Survey](https://mlsanity.com/paper/2404.19664/) (2024)
- [Knowledge Distillation on Graphs: A Survey](https://mlsanity.com/paper/2302.00219/) (2023)
- [Comprehensive Overview of Reward Engineering and Shaping in Advancing Reinforcement Learning Applications](https://mlsanity.com/paper/2408.10215/) (2024)
- [A Survey of Sim-to-Real Methods in RL: Progress, Prospects and Challenges with Foundation Models](https://mlsanity.com/paper/2502.13187/) (2025)
- [Trustworthy Reinforcement Learning Against Intrinsic Vulnerabilities: Robustness, Safety, and Generalizability](https://mlsanity.com/paper/2209.08025/) (2022)
- [Prompt to Transfer: Sim-to-Real Transfer for Traffic Signal Control with Prompt Learning](https://mlsanity.com/paper/2308.14284/) (2023)
- [From Machine Learning to Robotics: Challenges and Opportunities for Embodied Intelligence](https://mlsanity.com/paper/2110.15245/) (2021)
- [World4RL: Diffusion World Models for Policy Refinement with Reinforcement Learning for Robotic Manipulation](https://mlsanity.com/paper/2509.19080/) (2025)
- [Provably Safe Reinforcement Learning: Conceptual Analysis, Survey, and Benchmarking](https://mlsanity.com/paper/2205.06750/) (2022)
- [Towards Safe Reinforcement Learning with a Safety Editor Policy](https://mlsanity.com/paper/2201.12427/) (2022)
- [Scalable Multi-Agent Reinforcement Learning for Warehouse Logistics with Robotic and Human Co-Workers](https://mlsanity.com/paper/2212.11498/) (2022)
- [Evolutionary Reinforcement Learning: A Survey](https://mlsanity.com/paper/2303.04150/) (2023)
- [Probabilistic Adaptation of Text-to-Video Models](https://mlsanity.com/paper/2306.01872/) (2023)

## Abstract (arXiv)

Deep reinforcement learning has recently seen huge success across multiple areas in the robotics domain. Owing to the limitations of gathering real-world data, i.e., sample inefficiency and the cost of collecting it, simulation environments are utilized for training the different agents. This not only aids in providing a potentially infinite data source, but also alleviates safety concerns with real robots. Nonetheless, the gap between the simulated and real worlds degrades the performance of the policies once the models are transferred into real robots. Multiple research efforts are therefore now being directed towards closing this sim-to-real gap and accomplish more efficient policy transfer. Recent years have seen the emergence of multiple methods applicable to different domains, but there is a lack, to the best of our knowledge, of a comprehensive review summarizing and putting into context the different methods. In this survey paper, we cover the fundamental background behind sim-to-real transfer in deep reinforcement learning and overview the main methods being utilized at the moment: domain randomization, domain adaptation, imitation learning, meta-learning and knowledge distillation. We categorize some of the most relevant recent works, and outline the main application scenarios. Finally, we discuss the main opportunities and challenges of the different approaches and point to the most promising directions.