# Online vs. Offline Adaptive Domain Randomization Benchmark

arXiv: https://arxiv.org/abs/2206.14661 · HTML: https://mlsanity.com/paper/2206.14661/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 29 references that are in the map.
- First posted on arXiv 29 Jun 2022.

## Builds on (most cited first, 23 of 23)

- [OpenAI Gym](https://mlsanity.com/paper/1606.01540/) (2016)
- [Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World](https://mlsanity.com/paper/1703.06907/) (2017)
- [Solving Rubik's Cube with a Robot Hand](https://mlsanity.com/paper/1910.07113/) (2019)
- [Sim-to-Real Transfer of Robotic Control with Dynamics Randomization](https://mlsanity.com/paper/1710.06537/) (2017)
- [Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Survey](https://mlsanity.com/paper/2009.13303/) (2020)
- [Sim-to-Real: Learning Agile Locomotion For Quadruped Robots](https://mlsanity.com/paper/1804.10332/) (2018)
- [CAD2RL: Real Single-Image Flight without a Single Real Image](https://mlsanity.com/paper/1611.04201/) (2016)
- [Closing the Sim-to-Real Loop: Adapting Simulation Randomization with Real World Experience](https://mlsanity.com/paper/1810.05687/) (2018)
- [Active Domain Randomization](https://mlsanity.com/paper/1904.04762/) (2019)
- [EPOpt: Learning Robust Neural Network Policies Using Model Ensembles](https://mlsanity.com/paper/1610.01283/) (2016)
- [BayesSim: adaptive domain randomization via probabilistic inference for robotics simulators](https://mlsanity.com/paper/1906.01728/) (2019)
- [Understanding Domain Randomization for Sim-to-real Transfer](https://mlsanity.com/paper/2110.03239/) (2021)
- [Transferring End-to-End Visuomotor Control from Simulation to Real World for a Multi-Stage Task](https://mlsanity.com/paper/1707.02267/) (2017)
- [Robot Learning from Randomized Simulations: A Review](https://mlsanity.com/paper/2111.00956/) (2021)
- [Data-efficient Domain Randomization with Bayesian Optimization](https://mlsanity.com/paper/2003.02471/) (2020)
- [Assessing Transferability from Simulation to Reality for Reinforcement Learning](https://mlsanity.com/paper/1907.04685/) (2019)
- [How to pick the domain randomization parameters for sim-to-real transfer of reinforcement learning policies?](https://mlsanity.com/paper/1903.11774/) (2019)
- [DROPO: Sim-to-Real Transfer with Offline Domain Randomization](https://mlsanity.com/paper/2201.08434/) (2022)
- [Reinforcement Learning for Pivoting Task](https://mlsanity.com/paper/1703.00472/) (2017)
- [DROID: Minimizing the Reality Gap using Single-Shot Human Demonstration](https://mlsanity.com/paper/2102.11003/) (2021)
- [Sim-to-Real Transfer for Robotic Manipulation with Tactile Sensory](https://mlsanity.com/paper/2103.00410/) (2021)
- [A User's Guide to Calibrating Robotics Simulators](https://mlsanity.com/paper/2011.08985/) (2020)
- [Coarse-to-Fine for Sim-to-Real: Sub-Millimetre Precision Across Wide Task Spaces](https://mlsanity.com/paper/2105.11283/) (2021)

## Built on by (most cited first, 4 of 4)

- [Domain Randomization via Entropy Maximization](https://mlsanity.com/paper/2311.01885/) (2023)
- [Domain Randomization for Robust, Affordable and Effective Closed-loop Control of Soft Robots](https://mlsanity.com/paper/2303.04136/) (2023)
- [Pessimism Principle Can Be Effective: Towards a Framework for Zero-Shot Transfer Reinforcement Learning](https://mlsanity.com/paper/2505.18447/) (2025)
- [Shaping Laser Pulses with Reinforcement Learning](https://mlsanity.com/paper/2503.00499/) (2025)

## Abstract (arXiv)

Physics simulators have shown great promise for conveniently learning reinforcement learning policies in safe, unconstrained environments. However, transferring the acquired knowledge to the real world can be challenging due to the reality gap. To this end, several methods have been recently proposed to automatically tune simulator parameters with posterior distributions given real data, for use with domain randomization at training time. These approaches have been shown to work for various robotic tasks under different settings and assumptions. Nevertheless, existing literature lacks a thorough comparison of existing adaptive domain randomization methods with respect to transfer performance and real-data efficiency. In this work, we present an open benchmark for both offline and online methods (SimOpt, BayRn, DROID, DROPO), to shed light on which are most suitable for each setting and task at hand. We found that online methods are limited by the quality of the currently learned policy for the next iteration, while offline methods may sometimes fail when replaying trajectories in simulation with open-loop commands. The code used will be released at https://github.com/gabrieletiboni/adr-benchmark.