mlsanityData to 14 Oct 2026

Statistical Guarantees for Offline Domain Randomization

arXiv 2506.10133 ↗First posted 11 Jun 2025cs.LG cs.ROcited by 1

Not on the map. Why?

Abstract (arXiv)

Reinforcement-learning (RL) agents often struggle when deployed from simulation to the real-world. A dominant strategy for reducing the sim-to-real gap is domain randomization (DR) which trains the policy across many simulators produced by sampling dynamics parameters, but standard DR ignores offline data already available from the real system. We study offline domain randomization (ODR), which first fits a distribution over simulator parameters to an offline dataset. While a growing body of empirical work reports substantial gains with algorithms such as DROPO, the theoretical foundations of ODR remain largely unexplored. In this work, we cast ODR as a maximum-likelihood estimation over a parametric simulator family and provide statistical guarantees: under mild regularity and identifiability conditions, the estimator is weakly consistent (it converges in probability to the true dynamics as data grows), and it becomes strongly consistent (i.e., it converges almost surely to the true dynamics) when an additional uniform Lipschitz continuity assumption holds. We examine the practicality of these assumptions and outline relaxations that justify ODR's applicability across a broader range of settings. Taken together, our results place ODR on a principled footing and clarify when offline data can soundly guide the choice of a randomization distribution for downstream offline RL.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 24 of its 40 references that are in the map. First posted on arXiv 11 Jun 2025.

Builds on

24 of 40 references.

References · 24

  1. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  2. 2015Trust Region Policy Optimizationcited by 2,580
  3. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,920
  4. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,498
  5. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 743
  6. 2020Deep Reinforcement Learning for Autonomous Driving: A Surveycited by 721
  7. 2018Learning Dexterous In-Hand Manipulationcited by 687
  8. 2017Sim-to-Real Transfer of Robotic Control with Dynamics Randomizationcited by 672
  9. 2020Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Surveycited by 413
  10. 2018Sim-to-Real: Learning Agile Locomotion For Quadruped Robotscited by 319
  11. 2016CAD2RL: Real Single-Image Flight without a Single Real Imagecited by 233
  12. 2018Closing the Sim-to-Real Loop: Adapting Simulation Randomization with Real World Experiencecited by 217
12 more
  1. 2021Reinforcement learning based recommender systems: A surveycited by 188
  2. 2016EPOpt: Learning Robust Neural Network Policies Using Model Ensemblescited by 138
  3. 2016Sim-to-Real Robot Learning from Pixels with Progressive Netscited by 94
  4. 2019BayesSim: adaptive domain randomization via probabilistic inference for robotics simulatorscited by 85
  5. 2021Understanding Domain Randomization for Sim-to-real Transfercited by 80
  6. 2019Meta Reinforcement Learning for Sim-to-real Domain Adaptationcited by 49
  7. 2021A Survey of Deep Reinforcement Learning in Recommender Systems: A Systematic Review and Future Directionscited by 21
  8. 2022DROPO: Sim-to-Real Transfer with Offline Domain Randomizationcited by 19
  9. 2021DROID: Minimizing the Reality Gap using Single-Shot Human Demonstrationcited by 11
  10. 2021DR2L: Surfacing Corner Cases to Robustify Autonomous Driving via Domain Randomization Reinforcement Learningcited by 11
  11. 2019PAC Reinforcement Learning without Real-World Feedbackcited by 6
  12. 2019How Does an Approximate Model Help in Reinforcement Learning?cited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Mind the Sim-to-Real Gap & Think Like a Scientistcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2022Domain Generalization for Robust Model-Based Offline Reinforcement Learning
  2. 2024Provable Domain Adaptation for Offline Reinforcement Learning with Limited Samples
  3. 2024Integrating Domain Knowledge for handling Limited Data in Offline RL
  4. 2025On the Statistical Complexity for Offline and Low-Adaptive Reinforcement Learning with Structures
  5. 2021Behavioral Priors and Dynamics Models: Improving Performance and Domain Transfer in Offline RL
  6. 2024On Sample-Efficient Offline Reinforcement Learning: Data Diversity, Posterior Sampling, and Beyond
  7. 2022Regularizing a Model-based Policy Stationary Distribution to Stabilize Offline Reinforcement Learning
  8. 2024Benchmarks for Reinforcement Learning with Biased Offline Data and Imperfect Simulators