# ClearGrasp: 3D Shape Estimation of Transparent Objects for Manipulation

arXiv: https://arxiv.org/abs/1910.02550 · HTML: https://mlsanity.com/paper/1910.02550/

## Key facts

- Cited by 124 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Oct 2019.

## Built on by (most cited first, 30 of 124)

- [Rethinking Inductive Biases for Surface Normal Estimation](https://mlsanity.com/paper/2403.00712/) (2024)
- [Dex-NeRF: Using a Neural Radiance Field to Grasp Transparent Objects](https://mlsanity.com/paper/2110.14217/) (2021)
- [GraspNeRF: Multiview-based 6-DoF Grasp Detection for Transparent and Specular Objects Using Generalizable NeRF](https://mlsanity.com/paper/2210.06575/) (2022)
- [RGB-D Local Implicit Function for Depth Completion of Transparent Objects](https://mlsanity.com/paper/2104.00622/) (2021)
- [TransCG: A Large-Scale Real-World Dataset for Transparent Object Depth Completion and a Grasping Baseline](https://mlsanity.com/paper/2202.08471/) (2022)
- [Learning to See before Learning to Act: Visual Pre-training for Manipulation](https://mlsanity.com/paper/2107.00646/) (2021)
- [Domain Randomization-Enhanced Depth Simulation and Restoration for Perceiving and Grasping Specular and Transparent Objects](https://mlsanity.com/paper/2208.03792/) (2022)
- [ClearPose: Large-scale Transparent Object Dataset and Benchmark](https://mlsanity.com/paper/2203.03890/) (2022)
- [PhoCaL: A Multi-Modal Dataset for Category-Level Object Pose Estimation with Photometrically Challenging Objects](https://mlsanity.com/paper/2205.08811/) (2022)
- [StereOBJ-1M: Large-scale Stereo Image Dataset for 6D Object Pose Estimation](https://mlsanity.com/paper/2109.10115/) (2021)
- [Seeing Glass: Joint Point Cloud and Depth Completion for Transparent Objects](https://mlsanity.com/paper/2110.00087/) (2021)
- [Learning Depth Estimation for Transparent and Mirror Surfaces](https://mlsanity.com/paper/2307.15052/) (2023)
- [RTMV: A Ray-Traced Multi-View Synthetic Dataset for Novel View Synthesis](https://mlsanity.com/paper/2205.07058/) (2022)
- [Eikonal Fields for Refractive Novel-View Synthesis](https://mlsanity.com/paper/2202.00948/) (2022)
- [Diffusion Models for Monocular Depth Estimation: Overcoming Challenging Conditions](https://mlsanity.com/paper/2407.16698/) (2024)
- [Masked Depth Modeling for Spatial Perception](https://mlsanity.com/paper/2601.17895/) (2026)
- [Glass Segmentation with RGB-Thermal Image Pairs](https://mlsanity.com/paper/2204.05453/) (2022)
- [NViSII: A Scriptable Tool for Photorealistic Image Generation](https://mlsanity.com/paper/2105.13962/) (2021)
- [MonoGraspNet: 6-DoF Grasping with a Single RGB Image](https://mlsanity.com/paper/2209.13036/) (2022)
- [A Dense Material Segmentation Dataset for Indoor and Outdoor Scene Parsing](https://mlsanity.com/paper/2207.10614/) (2022)
- [Vision-driven Compliant Manipulation for Reliable, High-Precision Assembly Tasks](https://mlsanity.com/paper/2106.14070/) (2021)
- [SimNet: Enabling Robust Unknown Object Manipulation from Pure Synthetic Data via Stereo](https://mlsanity.com/paper/2106.16118/) (2021)
- [StereoPose: Category-Level 6D Transparent Object Pose Estimation from Stereo Images via Back-View NOCS](https://mlsanity.com/paper/2211.01644/) (2022)
- [Polarimetric Pose Prediction](https://mlsanity.com/paper/2112.03810/) (2021)
- [Transparent Shape from a Single View Polarization Image](https://mlsanity.com/paper/2204.06331/) (2022)
- [Hybrid Mesh-neural Representation for 3D Transparent Object Reconstruction](https://mlsanity.com/paper/2203.12613/) (2022)
- [TRansPose: Large-Scale Multispectral Dataset for Transparent Object](https://mlsanity.com/paper/2307.05016/) (2023)
- [MVTrans: Multi-View Perception of Transparent Objects](https://mlsanity.com/paper/2302.11683/) (2023)
- [TransNet: Category-Level Transparent Object Pose Estimation](https://mlsanity.com/paper/2208.10002/) (2022)
- [TODE-Trans: Transparent Object Depth Estimation with Transformer](https://mlsanity.com/paper/2209.08455/) (2022)

## Abstract (arXiv)

Transparent objects are a common part of everyday life, yet they possess unique visual properties that make them incredibly difficult for standard 3D sensors to produce accurate depth estimates for. In many cases, they often appear as noisy or distorted approximations of the surfaces that lie behind them. To address these challenges, we present ClearGrasp -- a deep learning approach for estimating accurate 3D geometry of transparent objects from a single RGB-D image for robotic manipulation. Given a single RGB-D image of transparent objects, ClearGrasp uses deep convolutional networks to infer surface normals, masks of transparent surfaces, and occlusion boundaries. It then uses these outputs to refine the initial depth estimates for all transparent surfaces in the scene. To train and test ClearGrasp, we construct a large-scale synthetic dataset of over 50,000 RGB-D images, as well as a real-world test benchmark with 286 RGB-D images of transparent objects and their ground truth geometries. The experiments demonstrate that ClearGrasp is substantially better than monocular depth estimation baselines and is capable of generalizing to real-world images and novel objects. We also demonstrate that ClearGrasp can be applied out-of-the-box to improve grasping algorithms' performance on transparent objects. Code, data, and benchmarks will be released. Supplementary materials available on the project website: https://sites.google.com/view/cleargrasp