# Multi-modal Transfer Learning for Grasping Transparent and Specular Objects

arXiv: https://arxiv.org/abs/2006.00028 · HTML: https://mlsanity.com/paper/2006.00028/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 May 2020.

## Built on by (most cited first, 13 of 13)

- [GraspNeRF: Multiview-based 6-DoF Grasp Detection for Transparent and Specular Objects Using Generalizable NeRF](https://mlsanity.com/paper/2210.06575/) (2022)
- [Mirror3D: Depth Refinement for Mirror Surfaces](https://mlsanity.com/paper/2106.06629/) (2021)
- [TransNet: Category-Level Transparent Object Pose Estimation](https://mlsanity.com/paper/2208.10002/) (2022)
- [On Robot Grasp Learning Using Equivariant Models](https://mlsanity.com/paper/2306.06489/) (2023)
- [Semi-supervised 3D Object Detection via Temporal Graph Neural Networks](https://mlsanity.com/paper/2202.00182/) (2022)
- [Self-supervised Transparent Liquid Segmentation for Robotic Pouring](https://mlsanity.com/paper/2203.01538/) (2022)
- [RFTrans: Leveraging Refractive Flow of Transparent Objects for Surface Normal Estimation and Manipulation](https://mlsanity.com/paper/2311.12398/) (2023)
- [Visual-tactile Fusion for Transparent Object Grasping in Complex Backgrounds](https://mlsanity.com/paper/2211.16693/) (2022)
- [LARG, Language-based Automatic Reward and Goal Generation](https://mlsanity.com/paper/2306.10985/) (2023)
- [STOPNet: Multiview-based 6-DoF Suction Detection for Transparent Objects on Production Lines](https://mlsanity.com/paper/2310.05717/) (2023)
- [TransNet: Transparent Object Manipulation Through Category-Level Pose Estimation](https://mlsanity.com/paper/2307.12400/) (2023)
- [Transparent Object Tracking with Enhanced Fusion Module](https://mlsanity.com/paper/2309.06701/) (2023)
- [Shaken, Not Stirred: A Novel Dataset for Visual Understanding of Glasses in Human-Robot Bartending Tasks](https://mlsanity.com/paper/2503.04308/) (2025)

## Abstract (arXiv)

State-of-the-art object grasping methods rely on depth sensing to plan robust grasps, but commercially available depth sensors fail to detect transparent and specular objects. To improve grasping performance on such objects, we introduce a method for learning a multi-modal perception model by bootstrapping from an existing uni-modal model. This transfer learning approach requires only a pre-existing uni-modal grasping model and paired multi-modal image data for training, foregoing the need for ground-truth grasp success labels nor real grasp attempts. Our experiments demonstrate that our approach is able to reliably grasp transparent and reflective objects. Video and supplementary material are available at https://sites.google.com/view/transparent-specular-grasping.