# How to pick the domain randomization parameters for sim-to-real transfer of reinforcement learning policies?

arXiv: https://arxiv.org/abs/1903.11774 · HTML: https://mlsanity.com/paper/1903.11774/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Mar 2019.

## Built on by (most cited first, 25 of 25)

- [DrEureka: Language Model Guided Sim-To-Real Transfer](https://mlsanity.com/paper/2406.01967/) (2024)
- [When to Trust Your Simulator: Dynamics-Aware Hybrid Offline-and-Online Reinforcement Learning](https://mlsanity.com/paper/2206.13464/) (2022)
- [Cross-Domain Policy Adaptation by Capturing Representation Mismatch](https://mlsanity.com/paper/2405.15369/) (2024)
- [ODRL: A Benchmark for Off-Dynamics Reinforcement Learning](https://mlsanity.com/paper/2410.20750/) (2024)
- [Domain Randomization via Entropy Maximization](https://mlsanity.com/paper/2311.01885/) (2023)
- [AdaptSim: Task-Driven Simulation Adaptation for Sim-to-Real Transfer](https://mlsanity.com/paper/2302.04903/) (2023)
- [H2O+: An Improved Framework for Hybrid Offline-and-Online RL with Dynamics Gaps](https://mlsanity.com/paper/2309.12716/) (2023)
- [A Comprehensive Survey of Cross-Domain Policy Transfer for Embodied Agents](https://mlsanity.com/paper/2402.04580/) (2024)
- [Automated Reinforcement Learning: An Overview](https://mlsanity.com/paper/2201.05000/) (2022)
- [Sim and Real: Better Together](https://mlsanity.com/paper/2110.00445/) (2021)
- [Marginalized Importance Sampling for Off-Environment Policy Evaluation](https://mlsanity.com/paper/2309.01807/) (2023)
- [TAR: Teacher-Aligned Representations via Contrastive Learning for Quadrupedal Locomotion](https://mlsanity.com/paper/2503.20839/) (2025)
- [Online vs. Offline Adaptive Domain Randomization Benchmark](https://mlsanity.com/paper/2206.14661/) (2022)
- [Feasible Adversarial Robust Reinforcement Learning for Underspecified Environments](https://mlsanity.com/paper/2207.09597/) (2022)
- [Active Reinforcement Learning for Robust Building Control](https://mlsanity.com/paper/2312.10289/) (2023)
- [Advancing Investment Frontiers: Industry-grade Deep Reinforcement Learning for Portfolio Optimization](https://mlsanity.com/paper/2403.07916/) (2024)
- [Domain Randomization for Robust, Affordable and Effective Closed-loop Control of Soft Robots](https://mlsanity.com/paper/2303.04136/) (2023)
- [Simplex-enabled Safe Continual Learning Machine](https://mlsanity.com/paper/2409.05898/) (2024)
- [Pessimism Principle Can Be Effective: Towards a Framework for Zero-Shot Transfer Reinforcement Learning](https://mlsanity.com/paper/2505.18447/) (2025)
- [Single RGB-D Camera Teleoperation for General Robotic Manipulation](https://mlsanity.com/paper/2106.14396/) (2021)
- [GBC: Generalized Behavior-Cloning Framework for Whole-Body Humanoid Imitation](https://mlsanity.com/paper/2508.09960/) (2025)
- [Cross-Domain Offline Policy Adaptation via Selective Transition Correction](https://mlsanity.com/paper/2602.05776/) (2026)
- [A Synthetic Dataset for Manometry Recognition in Robotic Applications](https://mlsanity.com/paper/2508.17468/) (2025)
- [Real-DRL: Teach and Learn in Reality](https://mlsanity.com/paper/2511.00112/) (2025)
- [How Should a Simulation-to-Reality Transfer Budget Be Spent?](https://mlsanity.com/paper/2606.22062/) (2026)

## Abstract (arXiv)

Recently, reinforcement learning (RL) algorithms have demonstrated remarkable success in learning complicated behaviors from minimally processed input. However, most of this success is limited to simulation. While there are promising successes in applying RL algorithms directly on real systems, their performance on more complex systems remains bottle-necked by the relative data inefficiency of RL algorithms. Domain randomization is a promising direction of research that has demonstrated impressive results using RL algorithms to control real robots. At a high level, domain randomization works by training a policy on a distribution of environmental conditions in simulation. If the environments are diverse enough, then the policy trained on this distribution will plausibly generalize to the real world. A human-specified design choice in domain randomization is the form and parameters of the distribution of simulated environments. It is unclear how to the best pick the form and parameters of this distribution and prior work uses hand-tuned distributions. This extended abstract demonstrates that the choice of the distribution plays a major role in the performance of the trained policies in the real world and that the parameter of this distribution can be optimized to maximize the performance of the trained policies in the real world