# Robot Learning from Randomized Simulations: A Review

arXiv: https://arxiv.org/abs/2111.00956 · HTML: https://mlsanity.com/paper/2111.00956/

## Key facts

- Cited by 63 papers in the map (data through 14 Oct 2026).
- Builds on 101 of its 195 references that are in the map.
- First posted on arXiv 1 Nov 2021.

## Builds on (most cited first, 30 of 101)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Adversarial examples in the physical world](https://mlsanity.com/paper/1607.02533/) (2016)
- [OpenAI Gym](https://mlsanity.com/paper/1606.01540/) (2016)
- [Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World](https://mlsanity.com/paper/1703.06907/) (2017)
- [Progressive Neural Networks](https://mlsanity.com/paper/1606.04671/) (2016)
- [A Comprehensive Survey on Transfer Learning](https://mlsanity.com/paper/1911.02685/) (2019)
- [Hindsight Experience Replay](https://mlsanity.com/paper/1707.01495/) (2017)
- [Adversarial Examples Are Not Bugs, They Are Features](https://mlsanity.com/paper/1905.02175/) (2019)
- [Solving Rubik's Cube with a Robot Hand](https://mlsanity.com/paper/1910.07113/) (2019)
- [Hamiltonian Neural Networks](https://mlsanity.com/paper/1906.01563/) (2019)
- [Learning Dexterous In-Hand Manipulation](https://mlsanity.com/paper/1808.00177/) (2018)
- [Sim-to-Real Transfer of Robotic Control with Dynamics Randomization](https://mlsanity.com/paper/1710.06537/) (2017)
- [Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methods](https://mlsanity.com/paper/1705.07263/) (2017)
- [Interaction Networks for Learning about Objects, Relations and Physics](https://mlsanity.com/paper/1612.00222/) (2016)
- [RMA: Rapid Motor Adaptation for Legged Robots](https://mlsanity.com/paper/2107.04034/) (2021)
- [The frontier of simulation-based inference](https://mlsanity.com/paper/1911.01429/) (2019)
- [Robust Adversarial Reinforcement Learning](https://mlsanity.com/paper/1703.02702/) (2017)
- [Policy Distillation](https://mlsanity.com/paper/1511.06295/) (2015)
- [Learning to reinforcement learn](https://mlsanity.com/paper/1611.05763/) (2016)
- [Adversarial Attacks on Neural Network Policies](https://mlsanity.com/paper/1702.02284/) (2017)
- [Sim-to-Real: Learning Agile Locomotion For Quadruped Robots](https://mlsanity.com/paper/1804.10332/) (2018)
- [Learning Hand-Eye Coordination for Robotic Grasping with Deep Learning and Large-Scale Data Collection](https://mlsanity.com/paper/1603.02199/) (2016)
- [Emergent Tool Use From Multi-Agent Autocurricula](https://mlsanity.com/paper/1909.07528/) (2019)

## Built on by (most cited first, 30 of 63)

- [Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learning](https://mlsanity.com/paper/2304.13653/) (2023)
- [Cosmos-Transfer1: Conditional World Generation with Adaptive Multimodal Control](https://mlsanity.com/paper/2503.14492/) (2025)
- [DrEureka: Language Model Guided Sim-To-Real Transfer](https://mlsanity.com/paper/2406.01967/) (2024)
- [GATSBI: Generative Adversarial Training for Simulation-Based Inference](https://mlsanity.com/paper/2203.06481/) (2022)
- [Sim-to-Lab-to-Real: Safe Reinforcement Learning with Shielding and Generalization Guarantees](https://mlsanity.com/paper/2201.08355/) (2022)
- [ReVLA: Reverting Visual Domain Limitation of Robotic Foundation Models](https://mlsanity.com/paper/2409.15250/) (2024)
- [On the Role of the Action Space in Robot Manipulation Learning and Sim-to-Real Transfer](https://mlsanity.com/paper/2312.03673/) (2023)
- [The Reality Gap in Robotics: Challenges, Solutions, and Best Practices](https://mlsanity.com/paper/2510.20808/) (2025)
- [ReBot: Scaling Robot Learning with Real-to-Sim-to-Real Robotic Video Synthesis](https://mlsanity.com/paper/2503.14526/) (2025)
- [Domain Randomization via Entropy Maximization](https://mlsanity.com/paper/2311.01885/) (2023)
- [AdaptSim: Task-Driven Simulation Adaptation for Sim-to-Real Transfer](https://mlsanity.com/paper/2302.04903/) (2023)
- [Transfer Learning in Robotics: An Upcoming Breakthrough? A Review of Promises and Challenges](https://mlsanity.com/paper/2311.18044/) (2023)
- [Learning Robot Soccer from Egocentric Vision with Deep Reinforcement Learning](https://mlsanity.com/paper/2405.02425/) (2024)
- [ORV: 4D Occupancy-centric Robot Video Generation](https://mlsanity.com/paper/2506.03079/) (2025)
- [Eurekaverse: Environment Curriculum Generation via Large Language Models](https://mlsanity.com/paper/2411.01775/) (2024)
- [Analysis of Randomization Effects on Sim2Real Transfer in Reinforcement Learning for Robotic Manipulation Tasks](https://mlsanity.com/paper/2206.06282/) (2022)
- [How Crucial is Transformer in Decision Transformer?](https://mlsanity.com/paper/2211.14655/) (2022)
- [High-Performance Reinforcement Learning on Spot: Optimizing Simulation Parameters with Distributional Measures](https://mlsanity.com/paper/2504.17857/) (2025)
- [Domain Randomization for Sim2real Transfer of Automatically Generated Grasping Datasets](https://mlsanity.com/paper/2310.04517/) (2023)
- [Continual Domain Randomization](https://mlsanity.com/paper/2403.12193/) (2024)
- [Tune to Learn: How Controller Gains Shape Robot Policy Learning](https://mlsanity.com/paper/2604.02523/) (2026)
- [Constrained Reinforcement Learning using Distributional Representation for Trustworthy Quadrotor UAV Tracking Control](https://mlsanity.com/paper/2302.11694/) (2023)
- [Learning Object Properties Using Robot Proprioception via Differentiable Robot-Object Interaction](https://mlsanity.com/paper/2410.03920/) (2024)
- [Policy Gradient for LQR with Domain Randomization](https://mlsanity.com/paper/2503.24371/) (2025)
- [Online vs. Offline Adaptive Domain Randomization Benchmark](https://mlsanity.com/paper/2206.14661/) (2022)
- [Stochastic optimal well control in subsurface reservoirs using reinforcement learning](https://mlsanity.com/paper/2207.03456/) (2022)
- [Reasoning in visual navigation of end-to-end trained agents: a dynamical systems approach](https://mlsanity.com/paper/2503.08306/) (2025)
- [Distributionally Robust Optimization via Diffusion Ambiguity Modeling](https://mlsanity.com/paper/2510.22757/) (2025)
- [AnyBipe: An End-to-End Framework for Training and Deploying Bipedal Robots Guided by Large Language Models](https://mlsanity.com/paper/2409.08904/) (2024)
- [DeFM: Learning Foundation Representations from Depth for Robotics](https://mlsanity.com/paper/2601.18923/) (2026)

## Abstract (arXiv)

The rise of deep learning has caused a paradigm shift in robotics research, favoring methods that require large amounts of data. Unfortunately, it is prohibitively expensive to generate such data sets on a physical platform. Therefore, state-of-the-art approaches learn in simulation where data generation is fast as well as inexpensive and subsequently transfer the knowledge to the real robot (sim-to-real). Despite becoming increasingly realistic, all simulators are by construction based on models, hence inevitably imperfect. This raises the question of how simulators can be modified to facilitate learning robot control policies and overcome the mismatch between simulation and reality, often called the 'reality gap'. We provide a comprehensive review of sim-to-real research for robotics, focusing on a technique named 'domain randomization' which is a method for learning from randomized simulations.