# Domain Randomization and Generative Models for Robotic Grasping

arXiv: https://arxiv.org/abs/1710.06425 · HTML: https://mlsanity.com/paper/1710.06425/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Oct 2017.

## Built on by (most cited first, 30 of 36)

- [BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learning](https://mlsanity.com/paper/2202.02005/) (2022)
- [Structured World Models from Human Videos](https://mlsanity.com/paper/2308.10901/) (2023)
- [Urban Driving with Conditional Imitation Learning](https://mlsanity.com/paper/1912.00177/) (2019)
- [Beyond Pick-and-Place: Tackling Robotic Stacking of Diverse Shapes](https://mlsanity.com/paper/2110.06192/) (2021)
- [Unity Perception: Generate Synthetic Data for Computer Vision](https://mlsanity.com/paper/2107.04259/) (2021)
- [GraspGen: A Diffusion-based Framework for 6-DOF Grasping with On-Generator Training](https://mlsanity.com/paper/2507.13097/) (2025)
- [HuB: Learning Extreme Humanoid Balance](https://mlsanity.com/paper/2505.07294/) (2025)
- [Object Detection Using Sim2Real Domain Randomization for Robotic Applications](https://mlsanity.com/paper/2208.04171/) (2022)
- [Markerless Camera-to-Robot Pose Estimation via Self-supervised Sim-to-Real Transfer](https://mlsanity.com/paper/2302.14332/) (2023)
- [Distributionally Robust Policy Learning via Adversarial Environment Generation](https://mlsanity.com/paper/2107.06353/) (2021)
- [FSPO: Few-Shot Optimization of Synthetic Preferences Personalizes to Real Users](https://mlsanity.com/paper/2502.19312/) (2025)
- [Back-to-Bones: Rediscovering the Role of Backbones in Domain Generalization](https://mlsanity.com/paper/2209.01121/) (2022)
- [DA2 Dataset: Toward Dexterity-Aware Dual-Arm Grasping](https://mlsanity.com/paper/2208.00408/) (2022)
- [Generalizing 6-DoF Grasp Detection via Domain Prior Knowledge](https://mlsanity.com/paper/2404.01727/) (2024)
- [LIFT: Reinforcement Learning in Computer Systems by Learning From Demonstrations](https://mlsanity.com/paper/1808.07903/) (2018)
- [Attribute-Based Robotic Grasping with Data-Efficient Adaptation](https://mlsanity.com/paper/2501.02149/) (2025)
- [Palm up: Playing in the Latent Manifold for Unsupervised Pretraining](https://mlsanity.com/paper/2210.10913/) (2022)
- [Ab Initio Particle-based Object Manipulation](https://mlsanity.com/paper/2107.08865/) (2021)
- [Towards Domain-agnostic Depth Completion](https://mlsanity.com/paper/2207.14466/) (2022)
- [Environment as Policy: Learning to Race in Unseen Tracks](https://mlsanity.com/paper/2410.22308/) (2024)
- [Learning to Efficiently Plan Robust Frictional Multi-Object Grasps](https://mlsanity.com/paper/2210.07420/) (2022)
- [Sim2Rec: A Simulator-based Decision-making Approach to Optimize Real-World Long-term User Engagement in Sequential Recommender Systems](https://mlsanity.com/paper/2305.04832/) (2023)
- [Sim-to-Real Grasp Detection with Global-to-Local RGB-D Adaptation](https://mlsanity.com/paper/2403.11511/) (2024)
- [Unsupervised Feature Learning for Manipulation with Contrastive Domain Randomization](https://mlsanity.com/paper/2103.11144/) (2021)
- [Distilled Domain Randomization](https://mlsanity.com/paper/2112.03149/) (2021)
- [Bayesian Optimisation for Robust Model Predictive Control under Model Parameter Uncertainty](https://mlsanity.com/paper/2203.00551/) (2022)
- [Synthetic Data and Simulators for Recommendation Systems: Current State and Future Directions](https://mlsanity.com/paper/2112.11022/) (2021)
- [Natural Language Robot Programming: NLP integrated with autonomous robotic grasping](https://mlsanity.com/paper/2304.02993/) (2023)
- [Stable-Sim2Real: Exploring Simulation of Real-Captured 3D Data with Two-Stage Depth Diffusion](https://mlsanity.com/paper/2507.23483/) (2025)
- [Stronger Generalization Guarantees for Robot Learning by Combining Generative Models and Real-World Data](https://mlsanity.com/paper/2111.08761/) (2021)

## Abstract (arXiv)

Deep learning-based robotic grasping has made significant progress thanks to algorithmic improvements and increased data availability. However, state-of-the-art models are often trained on as few as hundreds or thousands of unique object instances, and as a result generalization can be a challenge. In this work, we explore a novel data generation pipeline for training a deep neural network to perform grasp planning that applies the idea of domain randomization to object synthesis. We generate millions of unique, unrealistic procedurally generated objects, and train a deep neural network to perform grasp planning on these objects. Since the distribution of successful grasps for a given object can be highly multimodal, we propose an autoregressive grasp planning model that maps sensor inputs of a scene to a probability distribution over possible grasps. This model allows us to sample grasps efficiently at test time (or avoid sampling entirely). We evaluate our model architecture and data generation pipeline in simulation and the real world. We find we can achieve a >90% success rate on previously unseen realistic objects at test time in simulation despite having only been trained on random objects. We also demonstrate an 80% success rate on real-world grasp attempts despite having only been trained on random simulated objects.