mlsanityData to 14 Oct 2026

Domain Randomization via Entropy Maximization

arXiv 2311.01885 ↗First posted 3 Nov 2023cs.LG cs.ROcited by 19

Not on the map. Why?

Abstract (arXiv)

Varying dynamics parameters in simulation is a popular Domain Randomization (DR) approach for overcoming the reality gap in Reinforcement Learning (RL). Nevertheless, DR heavily hinges on the choice of the sampling distribution of the dynamics parameters, since high variability is crucial to regularize the agent's behavior but notoriously leads to overly conservative policies when randomizing excessively. In this paper, we propose a novel approach to address sim-to-real transfer, which automatically shapes dynamics distributions during training in simulation without requiring real-world data. We introduce DOmain RAndomization via Entropy MaximizatiON (DORAEMON), a constrained optimization problem that directly maximizes the entropy of the training distribution while retaining generalization capabilities. In achieving this, DORAEMON gradually increases the diversity of sampled dynamics parameters as long as the probability of success of the current policy is sufficiently high. We empirically validate the consistent benefits of DORAEMON in obtaining highly adaptive and generalizable policies, i.e. solving the task at hand across the widest range of dynamics parameters, as opposed to representative baselines from the DR literature. Notably, we also demonstrate the Sim2Real applicability of DORAEMON through its successful zero-shot transfer in a robotic manipulation setup under unknown real-world parameters.

Key facts. Cited by 19 papers in the map (data through 14 Oct 2026). Builds on 29 of its 38 references that are in the map. First posted on arXiv 3 Nov 2023.

Builds on

29 of 38 references.

References · 29

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  2. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,498
  3. 2019Solving Rubik's Cube with a Robot Handcited by 734
  4. 2017Sim-to-Real Transfer of Robotic Control with Dynamics Randomizationcited by 672
  5. 2020Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Surveycited by 413
  6. 2018Sim-to-Real: Learning Agile Locomotion For Quadruped Robotscited by 319
  7. 2017Automatic Goal Generation for Reinforcement Learning Agentscited by 259
  8. 2018Closing the Sim-to-Real Loop: Adapting Simulation Randomization with Real World Experiencecited by 217
  9. 2017Reverse Curriculum Generation for Reinforcement Learningcited by 198
  10. 2019Active Domain Randomizationcited by 170
  11. 2021Understanding Domain Randomization for Sim-to-real Transfercited by 80
  12. 2021Robot Learning from Randomized Simulations: A Reviewcited by 63
17 more
  1. 2017Markov Decision Processes with Continuous Side Informationcited by 56
  2. 2020Self-Paced Deep Reinforcement Learningcited by 50
  3. 2018Policy Transfer with Strategy Optimizationcited by 43
  4. 2019Assessing Transferability from Simulation to Reality for Reinforcement Learningcited by 33
  5. 2022Curriculum Reinforcement Learning using Optimal Transport via Gradual Domain Adaptationcited by 28
  6. 2019Self-Paced Contextual Reinforcement Learningcited by 26
  7. 2019How to pick the domain randomization parameters for sim-to-real transfer of reinforcement learning policies?cited by 25
  8. 2020Crossing The Gap: A Deep Dive into Zero-Shot Sim-to-Real Transfer for Dynamicscited by 21
  9. 2022DROPO: Sim-to-Real Transfer with Offline Domain Randomizationcited by 19
  10. 2017Reinforcement Learning for Pivoting Taskcited by 17
  11. 2018Accuracy-based Curriculum Learning in Deep Reinforcement Learningcited by 15
  12. 2023Outcome-directed Reinforcement Learning by Uncertainty & Temporal Distance-Aware Curriculum Goal Generationcited by 15
  13. 2022Analysis of Randomization Effects on Sim2Real Transfer in Reinforcement Learning for Robotic Manipulation Taskscited by 14
  14. 2021DROID: Minimizing the Reality Gap using Single-Shot Human Demonstrationcited by 11
  15. 2021Sim-to-Real Transfer for Robotic Manipulation with Tactile Sensorycited by 11
  16. 2022Online vs. Offline Adaptive Domain Randomization Benchmarkcited by 4
  17. 2023Reward-Machine-Guided, Self-Paced Reinforcement Learningcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 3; 2025: 11; 2026: 5.

19 papers.

Sort:

By area:

  1. 2024DrEureka: Language Model Guided Sim-To-Real Transfercited by 83
  2. 2025Fast and Robust: Task Sampling with Posterior and Diversity Synergies for Adaptive Decision-Makers in Randomized Environmentscited by 12
  3. 2025On Generalization Across Environments In Multi-Objective Reinforcement Learningcited by 8
  4. 2025One Net to Rule Them All: Domain Randomization in Quadcopter Racing Across Different Platformscited by 7
  5. 2025Morphologically Symmetric Reinforcement Learning for Ambidextrous Bimanual Manipulationcited by 5
  6. 2025Flow-based Domain Randomization for Learning and Sequencing Robotic Skillscited by 4
  7. 2024AnyBipe: An End-to-End Framework for Training and Deploying Bipedal Robots Guided by Large Language Modelscited by 3
  8. 2025Pessimism Principle Can Be Effective: Towards a Framework for Zero-Shot Transfer Reinforcement Learningcited by 2
  9. 2025Shaping Laser Pulses with Reinforcement Learningcited by 1
  10. 2025MULE: Multi-terrain and Unknown Load Adaptation for Effective Quadrupedal Locomotioncited by 1
  11. 2025Achieving Precise and Reliable Locomotion with Differentiable Simulation-Based System Identificationcited by 1
  12. 2024DyPNIPP: Predicting Environment Dynamics for RL-based Robust Informative Path Planningcited by 0
7 more
  1. 2025A Reinforcement Learning Approach to Non-prehensile Manipulation through Slidingcited by 0
  2. 2025Searching on a Budget: HW-NAS with 10 Latency Probescited by 0
  3. 2026Online Adaptive Reinforcement Learning with Echo State Networks for Non-Stationary Dynamicscited by 0
  4. 2026Sim-to-Real Transfer for Muscle-Actuated Robots via Generalized Actuator Networkscited by 0
  5. 2026Global Optimality for Constrained Exploration via Penalty Regularizationcited by 0
  6. 2026Bridging Risk Approximation Gaps in Model Predictive Task Sampling via In-Context Modelingcited by 0
  7. 2026XAI-Guided Fourier Domain Generalisation for Pure-Simulation Sim-to-Real Deep Reinforcement Learningcited by 0