mlsanityData to 7 Oct 2026

Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World

arXiv 1703.06907 ↗First posted 20 Mar 2017cs.RO cs.LGcited by 1,401

Not on the map. Why?

Cited by 88 in Offline RL and imitation learning from data, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Bridging the 'reality gap' that separates simulated robotics from experiments on hardware could accelerate robotic research through improved data availability. This paper explores domain randomization, a simple technique for training models on simulated images that transfer to real images by randomizing rendering in the simulator. With enough variability in the simulator, the real world may appear to the model as just another variation. We focus on the task of object localization, which is a stepping stone to general robotic manipulation skills. We find that it is possible to train a real-world object detector that is accurate to 1.5cm and robust to distractors and partial occlusions using only data from a simulator with non-realistic random textures. To demonstrate the capabilities of our detectors, we show they can be used to perform grasping in a cluttered environment. To our knowledge, this is the first successful transfer of a deep neural network trained only on simulated RGB images (without pre-training on real images) to the real world for the purpose of robotic control.

Key facts. Cited by 1,401 papers in the map (data through 7 Oct 2026). First posted on arXiv 20 Mar 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2018: 5; 2019: 2; 2020: 3; 2021: 222; 2022: 224; 2023: 192; 2024: 200; 2025: 249; 2026: 304.

1,401 papers.

Sort:

By area:

  1. 2021Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learningcited by 882
  2. 2021Generalizing to Unseen Domains: A Survey on Domain Generalizationcited by 762
  3. 2021Domain Generalization: A Surveycited by 760
  4. 2021Domain Generalization with MixStylecited by 563
  5. 2022Continual Test-Time Domain Adaptationcited by 520
  6. 2023Identifying the Risks of LM Agents with an LM-Emulated Sandboxcited by 514
  7. 2023End-to-end Autonomous Driving: Challenges and Frontierscited by 485
  8. 2024Evaluating Real-World Robot Manipulation Policies in Simulationcited by 484
  9. 2025RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulationcited by 477
  10. 2018Scalable agent alignment via reward modeling: a research directioncited by 438
  11. 2021Towards Out-Of-Distribution Generalization: A Surveycited by 431
  12. 2021RMA: Rapid Motor Adaptation for Legged Robotscited by 430
38 more
  1. 2021Omnidata: A Scalable Pipeline for Making Multi-Task Mid-Level Vision Datasets from 3D Scanscited by 336
  2. 2023MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrationscited by 308
  3. 2021A Survey of Embodied AI: From Simulators to Research Taskscited by 256
  4. 2021SynthSeg: Segmentation of brain MRI scans of any contrast and resolution without retrainingcited by 235
  5. 2023RoboGen: Towards Unleashing Infinite Data for Automated Robot Learning via Generative Simulationcited by 206
  6. 2021How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learnedcited by 201
  7. 2022Parameter-free Online Test-time Adaptationcited by 184
  8. 2023Scaling Robot Learning with Semantically Imagined Experiencecited by 175
  9. 2024Aligning Cyber Space with Physical World: A Comprehensive Survey on Embodied AIcited by 175
  10. 2025FoundationStereo: Zero-Shot Stereo Matchingcited by 168
  11. 2023RoboAgent: Generalization and Efficiency in Robot Manipulation via Semantic Augmentations and Action Chunkingcited by 167
  12. 2022Evolving Curricula with Regret-Based Environment Designcited by 166
  13. 2021A Survey of Zero-shot Generalisation in Deep Reinforcement Learningcited by 164
  14. 2021FSDR: Frequency Space Domain Randomization for Domain Generalizationcited by 163
  15. 2023Foundation Models for Decision Making: Problems, Methods, and Opportunitiescited by 160
  16. 2023Waymax: An Accelerated, Data-Driven Simulator for Large-Scale Autonomous Driving Researchcited by 158
  17. 2021A System for General In-Hand Object Re-Orientationcited by 155
  18. 2022SHIFT: A Synthetic Driving Dataset for Continuous Multi-Task Domain Adaptationcited by 146
  19. 2021Collaborating with Humans without Human Datacited by 139
  20. 2024Reconciling Reality through Simulation: A Real-to-Sim-to-Real Approach for Robust Manipulationcited by 135
  21. 2023Real-World Humanoid Locomotion with Reinforcement Learningcited by 132
  22. 2022Legged Locomotion in Challenging Terrains using Egocentric Visioncited by 131
  23. 2022Rapid Locomotion via Reinforcement Learningcited by 130
  24. 2024THE COLOSSEUM: A Benchmark for Evaluating Generalization for Robotic Manipulationcited by 128
  25. 2024Flow as the Cross-Domain Manipulation Interfacecited by 126
  26. 2018Reinforcement and Imitation Learning for Diverse Visuomotor Skillscited by 124
  27. 2021Contrastive Behavioral Similarity Embeddings for Generalization in Reinforcement Learningcited by 123
  28. 2023GenSim: Generating Robotic Simulation Tasks via Large Language Modelscited by 120
  29. 2021Predicting with Confidence on Unseen Distributionscited by 116
  30. 2021Data Collection and Quality Challenges in Deep Learning: A Data-Centric AI Perspectivecited by 116
  31. 2021Recurrent Model-Free RL Can Be a Strong Baseline for Many POMDPscited by 115
  32. 2021Counterfactual Generative Networkscited by 111
  33. 2021A Survey on Data Augmentation for Text Classificationcited by 111
  34. 2023Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learningcited by 111
  35. 2021Replay-Guided Adversarial Environment Designcited by 108
  36. 2024Scaling Cross-Embodied Learning: One Policy for Manipulation, Navigation, Locomotion and Aviationcited by 108
  37. 2022Reinforcement Learning with Action-Free Pre-Training from Videoscited by 107
  38. 2024URDFormer: A Pipeline for Constructing Articulated Simulation Environments from Real-World Imagescited by 106

50 most cited of 1,401 All 1,401 (JSON)