# Domain Randomization for Sim2real Transfer of Automatically Generated Grasping Datasets

arXiv: https://arxiv.org/abs/2310.04517 · HTML: https://mlsanity.com/paper/2310.04517/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 43 references that are in the map.
- First posted on arXiv 6 Oct 2023.

## Builds on (most cited first, 20 of 20)

- [Solving Rubik's Cube with a Robot Hand](https://mlsanity.com/paper/1910.07113/) (2019)
- [Illuminating search spaces by mapping elites](https://mlsanity.com/paper/1504.04909/) (2015)
- [RoboNet: Large-Scale Multi-Robot Learning](https://mlsanity.com/paper/1910.11215/) (2019)
- [Dex-Net 2.0: Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metrics](https://mlsanity.com/paper/1703.09312/) (2017)
- [How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learned](https://mlsanity.com/paper/2102.02915/) (2021)
- [ACRONYM: A Large-Scale Grasp Dataset Based on Simulation](https://mlsanity.com/paper/2011.09584/) (2020)
- [Benchmarking in Manipulation Research: The YCB Object and Model Set and Benchmarking Protocols](https://mlsanity.com/paper/1502.03143/) (2015)
- [Jacquard: A Large Scale Dataset for Robotic Grasp Detection](https://mlsanity.com/paper/1803.11469/) (2018)
- [From One Hand to Multiple Hands: Imitation Learning for Dexterous Manipulation from Single-Camera Teleoperation](https://mlsanity.com/paper/2204.12490/) (2022)
- [Deep Learning Approaches to Grasp Synthesis: A Review](https://mlsanity.com/paper/2207.02556/) (2022)
- [Robot Learning from Randomized Simulations: A Review](https://mlsanity.com/paper/2111.00956/) (2021)
- [Fast-Grasp'D: Dexterous Multi-finger Grasp Generation Through Differentiable Simulation](https://mlsanity.com/paper/2306.08132/) (2023)
- [SimGAN: Hybrid Simulator Identification for Domain Adaptation via Adversarial Reinforcement Learning](https://mlsanity.com/paper/2101.06005/) (2021)
- [Multi-FinGAN: Generative Coarse-To-Fine Sampling of Multi-Finger Grasps](https://mlsanity.com/paper/2012.09696/) (2020)
- [Reinforcement Learning for Pivoting Task](https://mlsanity.com/paper/1703.00472/) (2017)
- [Quantifying the Reality Gap in Robotic Manipulation Tasks](https://mlsanity.com/paper/1811.01484/) (2018)
- [Robotic Grasping from Classical to Modern: A Survey](https://mlsanity.com/paper/2202.03631/) (2022)
- [Automatic Acquisition of a Repertoire of Diverse Grasping Trajectories through Behavior Shaping and Novelty Search](https://mlsanity.com/paper/2205.08189/) (2022)
- [Quality Diversity under Sparse Reward and Sparse Interaction: Application to Grasping in Robotics](https://mlsanity.com/paper/2308.05483/) (2023)
- [Facilitating Sim-to-real by Intrinsic Stochasticity of Real-Time Simulation in Reinforcement Learning for Robot Manipulation](https://mlsanity.com/paper/2304.06056/) (2023)

## Built on by (most cited first, 9 of 9)

- [RoboGSim: A Real2Sim2Real Robotic Gaussian Splatting Simulator](https://mlsanity.com/paper/2411.11839/) (2024)
- [HuB: Learning Extreme Humanoid Balance](https://mlsanity.com/paper/2505.07294/) (2025)
- [Speeding up 6-DoF Grasp Sampling with Quality-Diversity](https://mlsanity.com/paper/2403.06173/) (2024)
- [QDGset: A Large Scale Grasping Dataset Generated with Quality-Diversity](https://mlsanity.com/paper/2410.02319/) (2024)
- [A Simulation Pipeline to Facilitate Real-World Robotic Reinforcement Learning Applications](https://mlsanity.com/paper/2502.15649/) (2025)
- [RoboPearls: Editable Video Simulation for Robot Manipulation](https://mlsanity.com/paper/2506.22756/) (2025)
- [Placeit! A Framework for Learning Robot Object Placement Skills](https://mlsanity.com/paper/2510.09267/) (2025)
- [Controlling Intent Expressiveness in Robot Motion with Diffusion Models](https://mlsanity.com/paper/2510.12370/) (2025)
- [Simulation-Driven Imitation Learning for Biosignals-Free Shared-Autonomy Prosthetic Grasping](https://mlsanity.com/paper/2606.07389/) (2026)

## Abstract (arXiv)

Robotic grasping refers to making a robotic system pick an object by applying forces and torques on its surface. Many recent studies use data-driven approaches to address grasping, but the sparse reward nature of this task made the learning process challenging to bootstrap. To avoid constraining the operational space, an increasing number of works propose grasping datasets to learn from. But most of them are limited to simulations. The present paper investigates how automatically generated grasps can be exploited in the real world. More than 7000 reach-and-grasp trajectories have been generated with Quality-Diversity (QD) methods on 3 different arms and grippers, including parallel fingers and a dexterous hand, and tested in the real world. Conducted analysis on the collected measure shows correlations between several Domain Randomization-based quality criteria and sim-to-real transferability. Key challenges regarding the reality gap for grasping have been identified, stressing matters on which researchers on grasping should focus in the future. A QD approach has finally been proposed for making grasps more robust to domain randomization, resulting in a transfer ratio of 84% on the Franka Research 3 arm.