# Multi-Task Domain Adaptation for Deep Learning of Instance Grasping from Simulation

arXiv: https://arxiv.org/abs/1710.06422 · HTML: https://mlsanity.com/paper/1710.06422/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Oct 2017.

## Built on by (most cited first, 30 of 36)

- [ManipulaTHOR: A Framework for Visual Object Manipulation](https://mlsanity.com/paper/2104.11213/) (2021)
- [Instance Segmentation in the Dark](https://mlsanity.com/paper/2304.14298/) (2023)
- [A Survey of Sim-to-Real Methods in RL: Progress, Prospects and Challenges with Foundation Models](https://mlsanity.com/paper/2502.13187/) (2025)
- [Prompt to Transfer: Sim-to-Real Transfer for Traffic Signal Control with Prompt Learning](https://mlsanity.com/paper/2308.14284/) (2023)
- [A Joint Modeling of Vision-Language-Action for Target-oriented Grasping in Clutter](https://mlsanity.com/paper/2302.12610/) (2023)
- [INVIGORATE: Interactive Visual Grounding and Grasping in Clutter](https://mlsanity.com/paper/2108.11092/) (2021)
- [Polybot: Training One Policy Across Robots While Embracing Variability](https://mlsanity.com/paper/2307.03719/) (2023)
- [Learning High-DOF Reaching-and-Grasping via Dynamic Representation of Gripper-Object Interaction](https://mlsanity.com/paper/2204.13998/) (2022)
- [Collision-Aware Target-Driven Object Grasping in Constrained Environments](https://mlsanity.com/paper/2104.00776/) (2021)
- [REGRAD: A Large-Scale Relational Grasp Dataset for Safe and Object-Specific Robotic Grasping in Clutter](https://mlsanity.com/paper/2104.14118/) (2021)
- [Weakly-Supervised Domain Adaptation of Deep Regression Trackers via Reinforced Knowledge Distillation](https://mlsanity.com/paper/2103.14496/) (2021)
- [Self-Supervised Interactive Object Segmentation Through a Singulation-and-Grasping Approach](https://mlsanity.com/paper/2207.09314/) (2022)
- [Attribute-Based Robotic Grasping with Data-Efficient Adaptation](https://mlsanity.com/paper/2501.02149/) (2025)
- [Domain-Adversarial and Conditional State Space Model for Imitation Learning](https://mlsanity.com/paper/2001.11628/) (2020)
- [Attribute-Based Robotic Grasping with One-Grasp Adaptation](https://mlsanity.com/paper/2104.02271/) (2021)
- [Practical Imitation Learning in the Real World via Task Consistency Loss](https://mlsanity.com/paper/2202.01862/) (2022)
- [DiAReL: Reinforcement Learning with Disturbance Awareness for Robust Sim2Real Policy Transfer in Robot Control](https://mlsanity.com/paper/2306.09010/) (2023)
- [PI-QT-Opt: Predictive Information Improves Multi-Task Robotic Reinforcement Learning at Scale](https://mlsanity.com/paper/2210.08217/) (2022)
- [Sim-to-Real Grasp Detection with Global-to-Local RGB-D Adaptation](https://mlsanity.com/paper/2403.11511/) (2024)
- [Arctique: An artificial histopathological dataset unifying realism and controllability for uncertainty quantification](https://mlsanity.com/paper/2411.07097/) (2024)
- [ADPro: a Test-time Adaptive Diffusion Policy via Manifold-constrained Denoising and Task-aware Initialization for Robotic Manipulation](https://mlsanity.com/paper/2508.06266/) (2025)
- [Invariance Co-training for Robot Visual Generalization](https://mlsanity.com/paper/2512.05230/) (2025)
- [SIM2REALVIZ: Visualizing the Sim2Real Gap in Robot Ego-Pose Estimation](https://mlsanity.com/paper/2109.11801/) (2021)
- [SafePicking: Learning Safe Object Extraction via Object-Level Mapping](https://mlsanity.com/paper/2202.05832/) (2022)
- [One-Shot Domain Adaptive and Generalizable Semantic Segmentation with Class-Aware Cross-Domain Transformers](https://mlsanity.com/paper/2212.07292/) (2022)
- [Uncertainty-aware Grounded Action Transformation towards Sim-to-Real Transfer for Traffic Signal Control](https://mlsanity.com/paper/2307.12388/) (2023)
- [Efficient Alignment of Unconditioned Action Prior for Language-conditioned Pick and Place in Clutter](https://mlsanity.com/paper/2503.09423/) (2025)
- [CoGrasp: 6-DoF Grasp Generation for Human-Robot Collaboration](https://mlsanity.com/paper/2210.03173/) (2022)
- [On the Challenges of Open World Recognitionunder Shifting Visual Domains](https://mlsanity.com/paper/2107.04461/) (2021)
- [SAGA: Open-World Mobile Manipulation via Structured Affordance Grounding](https://mlsanity.com/paper/2512.12842/) (2025)

## Abstract (arXiv)

Learning-based approaches to robotic manipulation are limited by the scalability of data collection and accessibility of labels. In this paper, we present a multi-task domain adaptation framework for instance grasping in cluttered scenes by utilizing simulated robot experiments. Our neural network takes monocular RGB images and the instance segmentation mask of a specified target object as inputs, and predicts the probability of successfully grasping the specified object for each candidate motor command. The proposed transfer learning framework trains a model for instance grasping in simulation and uses a domain-adversarial loss to transfer the trained model to real robots using indiscriminate grasping data, which is available both in simulation and the real world. We evaluate our model in real-world robot experiments, comparing it with alternative model architectures as well as an indiscriminate grasping baseline.