mlsanityData to 14 Oct 2026

Robot Learning from Randomized Simulations: A Review

arXiv 2111.00956 ↗First posted 1 Nov 2021cs.RO cs.LGcited by 63

Not on the map. Why?

Abstract (arXiv)

The rise of deep learning has caused a paradigm shift in robotics research, favoring methods that require large amounts of data. Unfortunately, it is prohibitively expensive to generate such data sets on a physical platform. Therefore, state-of-the-art approaches learn in simulation where data generation is fast as well as inexpensive and subsequently transfer the knowledge to the real robot (sim-to-real). Despite becoming increasingly realistic, all simulators are by construction based on models, hence inevitably imperfect. This raises the question of how simulators can be modified to facilitate learning robot control policies and overcome the mismatch between simulation and reality, often called the 'reality gap'. We provide a comprehensive review of sim-to-real research for robotics, focusing on a technique named 'domain randomization' which is a method for learning from randomized simulations.

Key facts. Cited by 63 papers in the map (data through 14 Oct 2026). Builds on 101 of its 195 references that are in the map. First posted on arXiv 1 Nov 2021.

Builds on

101 of 195 references.

References · 101

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  3. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  4. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  5. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  6. 2013Intriguing properties of neural networkscited by 5,155
  7. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  8. 2015Continuous control with deep reinforcement learningcited by 3,407
  9. 2016Adversarial examples in the physical worldcited by 1,769
  10. 2016OpenAI Gymcited by 1,736
  11. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,498
  12. 2016Progressive Neural Networkscited by 1,361
38 more
  1. 2019A Comprehensive Survey on Transfer Learningcited by 1,160
  2. 2017Hindsight Experience Replaycited by 1,136
  3. 2019Adversarial Examples Are Not Bugs, They Are Featurescited by 953
  4. 2019Solving Rubik's Cube with a Robot Handcited by 734
  5. 2019Hamiltonian Neural Networkscited by 712
  6. 2018Learning Dexterous In-Hand Manipulationcited by 687
  7. 2017Sim-to-Real Transfer of Robotic Control with Dynamics Randomizationcited by 672
  8. 2017Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methodscited by 546
  9. 2016Interaction Networks for Learning about Objects, Relations and Physicscited by 509
  10. 2021RMA: Rapid Motor Adaptation for Legged Robotscited by 508
  11. 2019The frontier of simulation-based inferencecited by 502
  12. 2017Robust Adversarial Reinforcement Learningcited by 451
  13. 2015Policy Distillationcited by 440
  14. 2016Learning to reinforcement learncited by 377
  15. 2017Adversarial Attacks on Neural Network Policiescited by 320
  16. 2018Sim-to-Real: Learning Agile Locomotion For Quadruped Robotscited by 319
  17. 2016Learning Hand-Eye Coordination for Robotic Grasping with Deep Learning and Large-Scale Data Collectioncited by 316
  18. 2019Emergent Tool Use From Multi-Agent Autocurriculacited by 313
  19. 2018Learning to Adapt in Dynamic, Real-World Environments Through Meta-Reinforcement Learningcited by 276
  20. 2020Learning Agile Robotic Locomotion Skills by Imitating Animalscited by 251
  21. 2017Asymmetric Actor Critic for Image-Based Robot Learningcited by 238
  22. 2016CAD2RL: Real Single-Image Flight without a Single Real Imagecited by 233
  23. 2017Distral: Robust Multitask Reinforcement Learningcited by 233
  24. 2018Sequential Neural Likelihood: Fast Likelihood-free Inference with Autoregressive Flowscited by 221
  25. 2019Automatic Posterior Transformation for Likelihood-Free Inferencecited by 219
  26. 2018Closing the Sim-to-Real Loop: Adapting Simulation Randomization with Real World Experiencecited by 217
  27. 2019Deep Lagrangian Networks: Using Physics as Model Prior for Deep Learningcited by 212
  28. 2014Robots that can adapt like animalscited by 204
  29. 2018Recasting Gradient-Based Meta-Learning as Hierarchical Bayescited by 180
  30. 2015Actor-Mimic: Deep Multitask and Transfer Reinforcement Learningcited by 178
  31. 2018Sim-to-Real via Sim-to-Sim: Data-efficient Robotic Grasping via Randomized-to-Canonical Adaptation Networkscited by 174
  32. 2019Active Domain Randomizationcited by 170
  33. 2017Flexible statistical inference for mechanistic models of neural dynamicscited by 163
  34. 2017Using Simulation and Domain Adaptation to Improve Efficiency of Deep Robotic Graspingcited by 144
  35. 2017Preparing for the Unknown: Learning a Universal Policy with Online System Identificationcited by 141
  36. 2016EPOpt: Learning Robust Neural Network Policies Using Model Ensemblescited by 138
  37. 2021Robust Reinforcement Learning on State Observations with Learned Optimal Adversarycited by 135
  38. 2018Sim-to-Real Reinforcement Learning for Deformable Object Manipulationcited by 131

50 most cited of 101 All 101 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 7; 2023: 13; 2024: 8; 2025: 15; 2026: 20.

63 papers.

Sort:

By area:

  1. 2023Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learningcited by 133
  2. 2025Cosmos-Transfer1: Conditional World Generation with Adaptive Multimodal Controlcited by 106
  3. 2024DrEureka: Language Model Guided Sim-To-Real Transfercited by 83
  4. 2022GATSBI: Generative Adversarial Training for Simulation-Based Inferencecited by 36
  5. 2022Sim-to-Lab-to-Real: Safe Reinforcement Learning with Shielding and Generalization Guaranteescited by 27
  6. 2024ReVLA: Reverting Visual Domain Limitation of Robotic Foundation Modelscited by 27
  7. 2023On the Role of the Action Space in Robot Manipulation Learning and Sim-to-Real Transfercited by 26
  8. 2025The Reality Gap in Robotics: Challenges, Solutions, and Best Practicescited by 24
  9. 2025ReBot: Scaling Robot Learning with Real-to-Sim-to-Real Robotic Video Synthesiscited by 22
  10. 2023Domain Randomization via Entropy Maximizationcited by 19
  11. 2023AdaptSim: Task-Driven Simulation Adaptation for Sim-to-Real Transfercited by 18
  12. 2023Transfer Learning in Robotics: An Upcoming Breakthrough? A Review of Promises and Challengescited by 18
38 more
  1. 2024Learning Robot Soccer from Egocentric Vision with Deep Reinforcement Learningcited by 17
  2. 2025ORV: 4D Occupancy-centric Robot Video Generationcited by 17
  3. 2024Eurekaverse: Environment Curriculum Generation via Large Language Modelscited by 16
  4. 2022Analysis of Randomization Effects on Sim2Real Transfer in Reinforcement Learning for Robotic Manipulation Taskscited by 14
  5. 2022How Crucial is Transformer in Decision Transformer?cited by 14
  6. 2025High-Performance Reinforcement Learning on Spot: Optimizing Simulation Parameters with Distributional Measurescited by 10
  7. 2023Domain Randomization for Sim2real Transfer of Automatically Generated Grasping Datasetscited by 9
  8. 2024Continual Domain Randomizationcited by 8
  9. 2026Tune to Learn: How Controller Gains Shape Robot Policy Learningcited by 8
  10. 2023Constrained Reinforcement Learning using Distributional Representation for Trustworthy Quadrotor UAV Tracking Controlcited by 5
  11. 2024Learning Object Properties Using Robot Proprioception via Differentiable Robot-Object Interactioncited by 5
  12. 2025Policy Gradient for LQR with Domain Randomizationcited by 5
  13. 2022Online vs. Offline Adaptive Domain Randomization Benchmarkcited by 4
  14. 2022Stochastic optimal well control in subsurface reservoirs using reinforcement learningcited by 4
  15. 2025Reasoning in visual navigation of end-to-end trained agents: a dynamical systems approachcited by 4
  16. 2025Distributionally Robust Optimization via Diffusion Ambiguity Modelingcited by 4
  17. 2024AnyBipe: An End-to-End Framework for Training and Deploying Bipedal Robots Guided by Large Language Modelscited by 3
  18. 2026DeFM: Learning Foundation Representations from Depth for Roboticscited by 3
  19. 2026Distributionally Robust Optimization via Generative Ambiguity Modelingcited by 3
  20. 2023Hindsight States: Blending Sim and Real Task Elements for Efficient Reinforcement Learningcited by 2
  21. 2023Domain Randomization for Robust, Affordable and Effective Closed-loop Control of Soft Robotscited by 2
  22. 2025Safe Continual Domain Adaptation after Sim2Real Transfer of Reinforcement Learning Policies in Roboticscited by 2
  23. 2025Bio-Inspired Plastic Neural Networks for Zero-Shot Out-of-Distribution Generalization in Complex Animal-Inspired Robotscited by 1
  24. 2025Post-Convergence Sim-to-Real Policy Transfer: A Principled Alternative to Cherry-Pickingcited by 1
  25. 2025Mitigating Goal Misgeneralization via Minimax Regretcited by 1
  26. 2025Control Synthesis with Reinforcement Learning: A Modeling Perspectivecited by 1
  27. 2026CoEnv: Driving Embodied Multi-Agent Collaboration via Compositional Environmentcited by 1
  28. 2022GPU-Accelerated Policy Optimization via Batch Automatic Differentiation of Gaussian Processes for Real-World Controlcited by 0
  29. 2023Sim-Anchored Learning for On-the-Fly Adaptationcited by 0
  30. 2023Augmenting Control over Exploration Space in Molecular Dynamics Simulators to Streamline De Novo Analysis through Generative Control Policiescited by 0
  31. 2023Pseudo-Likelihood Inferencecited by 0
  32. 2023Reinforcement Learning-Based Bionic Reflex Control for Anthropomorphic Robotic Grasping exploiting Domain Randomizationcited by 0
  33. 2024Physics-model-guided Worst-case Sampling for Safe Reinforcement Learningcited by 0
  34. 2025RM-RL: Role-Model Reinforcement Learning for Precise Robot Manipulationcited by 0
  35. 2025Posterior-driven Heuristic Support Adaptation in a Probabilistic Treatment of Real2Sim2Real for Vision-Driven Deformable Linear Object Manipulationcited by 0
  36. 2026Online Adaptive Reinforcement Learning with Echo State Networks for Non-Stationary Dynamicscited by 0
  37. 2026Developing AI Agents with Simulated Data: Why, what, and how?cited by 0
  38. 2026ManiDreams: An Open-Source Library for Robust Object Manipulation via Uncertainty-aware Task-specific Intuitive Physicscited by 0

50 most cited of 63 All 63 (JSON)