mlsanityData to 7 Oct 2026

Understanding Domain Randomization for Sim-to-real Transfer

arXiv 2110.03239 ↗First posted 7 Oct 2021cs.LGcited by 69

Not on the map. Why?

Abstract (arXiv)

Reinforcement learning encounters many challenges when applied directly in the real world. Sim-to-real transfer is widely used to transfer the knowledge learned from simulation to the real world. Domain randomization -- one of the most popular algorithms for sim-to-real transfer -- has been demonstrated to be effective in various tasks in robotics and autonomous driving. Despite its empirical successes, theoretical understanding on why this simple algorithm works is limited. In this paper, we propose a theoretical framework for sim-to-real transfers, in which the simulator is modeled as a set of MDPs with tunable parameters (corresponding to unknown physical parameters such as friction). We provide sharp bounds on the sim-to-real gap -- the difference between the value of policy returned by domain randomization and the value of an optimal policy for the real world. We prove that sim-to-real transfer can succeed under mild conditions without any real-world training samples. Our theory also highlights the importance of using memory (i.e., history-dependent policies) in domain randomization. Our proof is based on novel techniques that reduce the problem of bounding the sim-to-real gap to the problem of designing efficient learning algorithms for infinite-horizon MDPs, which we believe are of independent interest.

Key facts. Cited by 69 papers in the map (data through 7 Oct 2026). Builds on 34 of its 62 references that are in the map. First posted on arXiv 7 Oct 2021.

Builds on

34 of 62 references.

References · 34

  1. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  2. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,401
  3. 2018Learning Dexterous In-Hand Manipulationcited by 643
  4. 2017Sim-to-Real Transfer of Robotic Control with Dynamics Randomizationcited by 616
  5. 2019Provably Efficient Reinforcement Learning with Linear Function Approximationcited by 514
  6. 2018Is Q-learning Provably Efficient?cited by 485
  7. 2019Relay Policy Learning: Solving Long-Horizon Tasks via Imitation and Reinforcement Learningcited by 380
  8. 2018Sim-to-Real: Learning Agile Locomotion For Quadruped Robotscited by 277
  9. 2020Model-Based Reinforcement Learning with Value-Targeted Regressioncited by 233
  10. 2016CAD2RL: Real Single-Image Flight without a Single Real Imagecited by 227
  11. 2021Bellman Eluder Dimension: New Rich Classes of RL Problems, and Sample-Efficient Algorithmscited by 222
  12. 2018Closing the Sim-to-Real Loop: Adapting Simulation Randomization with Real World Experiencecited by 203
22 more
  1. 2020Nearly Minimax Optimal Reinforcement Learning for Linear Mixture Markov Decision Processescited by 199
  2. 2017Using Simulation and Domain Adaptation to Improve Efficiency of Deep Robotic Graspingcited by 138
  3. 2018Sim-to-Real Reinforcement Learning for Deformable Object Manipulationcited by 124
  4. 2014Model-based Reinforcement Learning and the Eluder Dimensioncited by 116
  5. 2020Learning to Generalize Across Long-Horizon Tasks from Human Demonstrationscited by 114
  6. 2016Sim-to-Real Robot Learning from Pixels with Progressive Netscited by 89
  7. 2019Model-free Reinforcement Learning in Infinite-horizon Average-reward Markov Decision Processescited by 81
  8. 2019Provably Efficient Q-Learning with Low Switching Costcited by 77
  9. 2019Sim2real transfer learning for 3D human pose estimation: motion to the rescuecited by 75
  10. 2021RL for Latent MDPs: Regret Guarantees and a Lower Boundcited by 69
  11. 2020Sample-Efficient Reinforcement Learning of Undercomplete POMDPscited by 66
  12. 2018Efficient Bias-Span-Constrained Exploration-Exploitation in Reinforcement Learningcited by 65
  13. 2016Transfer from Simulation to Real World through Learning Deep Inverse Dynamics Modelcited by 53
  14. 2020Learning Infinite-horizon Average-reward MDPs with Linear Function Approximationcited by 48
  15. 2019Regret Minimization for Reinforcement Learning by Evaluating the Optimal Bias Functioncited by 43
  16. 2018Policy Transfer with Strategy Optimizationcited by 41
  17. 2021Sublinear Regret for Learning POMDPscited by 19
  18. 2020Improved Analysis of UCRL2 with Empirical Bernstein Inequalitycited by 18
  19. 2020Modeling Long-horizon Tasks as Sequential Interaction Landscapescited by 14
  20. 2021DR2L: Surfacing Corner Cases to Robustify Autonomous Driving via Domain Randomization Reinforcement Learningcited by 10
  21. 2019PAC Reinforcement Learning without Real-World Feedbackcited by 4
  22. 2017Bayesian Optimization with Automatic Prior Selection for Data-Efficient Direct Policy Searchcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 5; 2023: 14; 2024: 9; 2025: 25; 2026: 15.

69 papers.

Sort:

By area:

  1. 2022Human-in-the-loop: Provably Efficient Preference-based Reinforcement Learning with General Function Approximationcited by 80
  2. 2023SPOC: Imitating Shortest Paths in Simulation Enables Effective Navigation and Manipulation in the Real Worldcited by 47
  3. 2025A Survey of Sim-to-Real Methods in RL: Progress, Prospects and Challenges with Foundation Modelscited by 36
  4. 2023Improving the Generalization of Segmentation Foundation Model under Distribution Shift via Weakly Supervised Adaptationcited by 31
  5. 2022A Comprehensive Survey of Data Augmentation in Visual Reinforcement Learningcited by 23
  6. 2023Diagnosis, Feedback, Adaptation: A Human-in-the-Loop Framework for Test-Time Policy Adaptationcited by 16
  7. 2023Domain Randomization via Entropy Maximizationcited by 15
  8. 2024The One RING: a Robotic Indoor Navigation Generalistcited by 15
  9. 2025Generative Models in Decision Making: A Surveycited by 14
  10. 2023Reward-Free Curricula for Training Robust World Modelscited by 13
  11. 2024A CMDP-within-online framework for Meta-Safe Reinforcement Learningcited by 12
  12. 2021Adjacency constraint for efficient hierarchical reinforcement learningcited by 10
38 more
  1. 2022Provable Sim-to-real Transfer in Continuous Domain with Partial Observationscited by 9
  2. 2023Bridging the Sim-to-Real Gap from the Information Bottleneck Perspectivecited by 8
  3. 2024Hybrid Transfer Reinforcement Learning: Provable Sample Efficiency from Shifted-Dynamics Datacited by 8
  4. 2025SPiDR: A Simple Approach for Zero-Shot Safety in Sim-to-Real Transfercited by 8
  5. 2023Deep Model Predictive Optimizationcited by 7
  6. 2023Domain Adaptive Sim-to-Real Segmentation of Oropharyngeal Organs Towards Robot-assisted Intubationcited by 6
  7. 2024Sample-efficient Learning of Infinite-horizon Average-reward MDPs with General Function Approximationcited by 6
  8. 2026Paying Less Generalization Tax: A Cross-Domain Generalization Study of RL Training for LLM Agentscited by 6
  9. 2026Quantifying the Utility of User Simulators for Building Collaborative LLM Assistantscited by 6
  10. 2024Test-Time Regret Minimization in Meta Reinforcement Learningcited by 5
  11. 2022Online vs. Offline Adaptive Domain Randomization Benchmarkcited by 4
  12. 2023Sample Efficient Reinforcement Learning with Partial Dynamics Knowledgecited by 4
  13. 2024Advancing Investment Frontiers: Industry-grade Deep Reinforcement Learning for Portfolio Optimizationcited by 4
  14. 2024OMPO: A Unified Framework for RL under Policy and Dynamics Shiftscited by 4
  15. 2025Flow-based Domain Randomization for Learning and Sequencing Robotic Skillscited by 4
  16. 2025Invariance Co-training for Robot Visual Generalizationcited by 4
  17. 2023Active Reinforcement Learning for Robust Building Controlcited by 3
  18. 2025Phys2Real: Fusing VLM Priors with Interactive Online Adaptation for Uncertainty-Aware Sim-to-Real Manipulationcited by 3
  19. 2026World Models: A Comprehensive Survey of Architectures, Methodologies, Reasoning Paradigms, and Applicationscited by 3
  20. 2022Vehicle Type Specific Waypoint Generationcited by 2
  21. 2023Online Nonstochastic Model-Free Reinforcement Learningcited by 2
  22. 2025Humans Coexist, So Must Embodied Artificial Agentscited by 2
  23. 2025Synthetic data enables context-aware bioacoustic sound event detectioncited by 2
  24. 2025Pessimism Principle Can Be Effective: Towards a Framework for Zero-Shot Transfer Reinforcement Learningcited by 2
  25. 2025MetaSTNet: Multimodal Meta-learning for Cellular Traffic Conformal Predictioncited by 2
  26. 2025PolySim: Bridging the Sim-to-Real Gap for Humanoid Control via Multi-Simulator Dynamics Randomizationcited by 2
  27. 2025Learning more with the same effort: how randomization improves the robustness of a robotic deep reinforcement learning agentcited by 1
  28. 2025Shaping Laser Pulses with Reinforcement Learningcited by 1
  29. 2025A Classification View on Meta Learning Banditscited by 1
  30. 2025Model-based controller assisted domain randomization for transient vibration suppression of nonlinear powertrain system with parametric uncertaintycited by 1
  31. 2025SynSpill: Improved Industrial Spill Detection With Synthetic Datacited by 1
  32. 2025A Multi-Modality Evaluation of the Reality Gap in Autonomous Driving Systemscited by 1
  33. 2025Blindfolded Experts Generalize Better: Insights from Robotic Manipulation and Videogamescited by 1
  34. 2026Constrained Meta Reinforcement Learning with Provable Test-Time Safetycited by 1
  35. 2026Reward-Conditioned Reinforcement Learningcited by 1
  36. 2023On Practical Robust Reinforcement Learning: Practical Uncertainty Set and Double-Agent Algorithmcited by 0
  37. 2023Domain Adaptive Sim-to-Real Segmentation of Oropharyngeal Organscited by 0
  38. 2023Sim-to-Real Transfer of Adaptive Control Parameters for AUV Stabilization under Current Disturbancecited by 0

50 most cited of 69 All 69 (JSON)