mlsanityData to 7 Oct 2026

DROPO: Sim-to-Real Transfer with Offline Domain Randomization

arXiv 2201.08434 ↗First posted 20 Jan 2022cs.RO cs.LGcited by 16

Not on the map. Why?

Abstract (arXiv)

In recent years, domain randomization over dynamics parameters has gained a lot of traction as a method for sim-to-real transfer of reinforcement learning policies in robotic manipulation; however, finding optimal randomization distributions can be difficult. In this paper, we introduce DROPO, a novel method for estimating domain randomization distributions for safe sim-to-real transfer. Unlike prior work, DROPO only requires a limited, precollected offline dataset of trajectories, and explicitly models parameter uncertainty to match real data using a likelihood-based approach. We demonstrate that DROPO is capable of recovering dynamic parameter distributions in simulation and finding a distribution capable of compensating for an unmodeled phenomenon. We also evaluate the method in two zero-shot sim-to-real transfer scenarios, showing successful domain transfer and improved performance over prior methods.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Builds on 30 of its 43 references that are in the map. First posted on arXiv 20 Jan 2022.

Builds on

30 of 43 references.

References · 30

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  3. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  4. 2016OpenAI Gymcited by 1,684
  5. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,401
  6. 2015End-to-End Training of Deep Visuomotor Policiescited by 847
  7. 2019Stabilizing Off-Policy Q-Learning via Bootstrapping Error Reductioncited by 833
  8. 2018Learning Dexterous In-Hand Manipulationcited by 643
  9. 2017Sim-to-Real Transfer of Robotic Control with Dynamics Randomizationcited by 616
  10. 2020Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Surveycited by 388
  11. 2018Sim-to-Real: Learning Agile Locomotion For Quadruped Robotscited by 277
  12. 2018Learning to Adapt in Dynamic, Real-World Environments Through Meta-Reinforcement Learningcited by 270
18 more
  1. 2016CAD2RL: Real Single-Image Flight without a Single Real Imagecited by 227
  2. 2018Closing the Sim-to-Real Loop: Adapting Simulation Randomization with Real World Experiencecited by 203
  3. 2019Active Domain Randomizationcited by 161
  4. 2016EPOpt: Learning Robust Neural Network Policies Using Model Ensemblescited by 136
  5. 2019BayesSim: adaptive domain randomization via probabilistic inference for robotics simulatorscited by 78
  6. 2019Meta Reinforcement Learning for Sim-to-real Domain Adaptationcited by 49
  7. 2021Auto-Tuned Sim-to-Real Transfercited by 45
  8. 2020Data-efficient Domain Randomization with Bayesian Optimizationcited by 43
  9. 2017Deep Predictive Policy Training using Reinforcement Learningcited by 30
  10. 2019Assessing Transferability from Simulation to Reality for Reinforcement Learningcited by 29
  11. 2020Crossing The Gap: A Deep Dive into Zero-Shot Sim-to-Real Transfer for Dynamicscited by 20
  12. 2017Sim2Real View Invariant Visual Servoing by Recurrent Controlcited by 19
  13. 2019Affordance Learning for End-to-End Visuomotor Robot Controlcited by 18
  14. 2017Reinforcement Learning for Pivoting Taskcited by 17
  15. 2021DROID: Minimizing the Reality Gap using Single-Shot Human Demonstrationcited by 11
  16. 2020A User's Guide to Calibrating Robotics Simulatorscited by 9
  17. 2020Few-shot model-based adaptation in noisy conditionscited by 4
  18. 2021Domain Curiosity: Learning Efficient Data Collection Strategies for Domain Adaptationcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 3; 2024: 0; 2025: 8; 2026: 4.

16 papers.

Sort:

By area:

  1. 2025The Reality Gap in Robotics: Challenges, Solutions, and Best Practicescited by 22
  2. 2023Domain Randomization via Entropy Maximizationcited by 15
  3. 2023Constrained Reinforcement Learning using Distributional Representation for Trustworthy Quadrotor UAV Tracking Controlcited by 5
  4. 2022Online vs. Offline Adaptive Domain Randomization Benchmarkcited by 4
  5. 2025Self-evolved Imitation Learning in Simulated Worldcited by 4
  6. 2025TAR: Teacher-Aligned Representations via Contrastive Learning for Quadrupedal Locomotioncited by 3
  7. 2023Domain Randomization for Robust, Affordable and Effective Closed-loop Control of Soft Robotscited by 2
  8. 2025Shaping Laser Pulses with Reinforcement Learningcited by 1
  9. 2025Safe Continual Domain Adaptation after Sim2Real Transfer of Reinforcement Learning Policies in Roboticscited by 1
  10. 2025Belief-Based Offline Reinforcement Learning for Delay-Robust Policy Optimizationcited by 1
  11. 2025Statistical Guarantees for Offline Domain Randomizationcited by 0
  12. 2025DiSA-IQL: Offline Reinforcement Learning for Robust Soft Robot Control under Distribution Shiftscited by 0
4 more
  1. 2026Grounding Sim-to-Real Generalization in Robotic Manipulation: An Empirical Study with Vision-Language-Action Modelscited by 0
  2. 2026Wiggle and Go! System Identification for Zero-Shot Dynamic Rope Manipulationcited by 0
  3. 2026A Unified Causal-Origin Taxonomy of Distributional Shifts in Reinforcement Learningcited by 0
  4. 2026How Should a Simulation-to-Reality Transfer Budget Be Spent?cited by 0