# TODE-Trans: Transparent Object Depth Estimation with Transformer

arXiv: https://arxiv.org/abs/2209.08455 · HTML: https://mlsanity.com/paper/2209.08455/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 33 references that are in the map.
- First posted on arXiv 18 Sep 2022.

## Builds on (most cited first, 24 of 24)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfer](https://mlsanity.com/paper/1907.01341/) (2019)
- [Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers](https://mlsanity.com/paper/2012.15840/) (2020)
- [Depth Map Prediction from a Single Image using a Multi-Scale Deep Network](https://mlsanity.com/paper/1406.2283/) (2014)
- [UNETR: Transformers for 3D Medical Image Segmentation](https://mlsanity.com/paper/2103.10504/) (2021)
- [Deep Ordinal Regression Network for Monocular Depth Estimation](https://mlsanity.com/paper/1806.02446/) (2018)
- [Deeper Depth Prediction with Fully Convolutional Residual Networks](https://mlsanity.com/paper/1606.00373/) (2016)
- [Non-Local Spatial Propagation Network for Depth Completion](https://mlsanity.com/paper/2007.10042/) (2020)
- [Single-Image Depth Perception in the Wild](https://mlsanity.com/paper/1604.03901/) (2016)
- [Sparse-to-Dense: Depth Prediction from Sparse Depth Samples and a Single Image](https://mlsanity.com/paper/1709.07492/) (2017)
- [DeepLiDAR: Deep Surface Normal Guided Depth Prediction for Outdoor Scene from Sparse LiDAR Data and Single Color Image](https://mlsanity.com/paper/1812.00488/) (2018)
- [ClearGrasp: 3D Shape Estimation of Transparent Objects for Manipulation](https://mlsanity.com/paper/1910.02550/) (2019)
- [Deep Depth Completion of a Single RGB-D Image](https://mlsanity.com/paper/1803.09326/) (2018)
- [Intel RealSense Stereoscopic Depth Cameras](https://mlsanity.com/paper/1705.05548/) (2017)
- [Learning Joint 2D-3D Representations for Depth Completion](https://mlsanity.com/paper/2012.12402/) (2020)
- [RGB-D Local Implicit Function for Depth Completion of Transparent Objects](https://mlsanity.com/paper/2104.00622/) (2021)
- [TransCG: A Large-Scale Real-World Dataset for Transparent Object Depth Completion and a Grasping Baseline](https://mlsanity.com/paper/2202.08471/) (2022)
- [Full 3D Reconstruction of Transparent Objects](https://mlsanity.com/paper/1805.03482/) (2018)
- [When Transformer Meets Robotic Grasping: Exploits Context for Efficient Grasp Detection](https://mlsanity.com/paper/2202.11911/) (2022)

## Built on by (most cited first, 13 of 13)

- [ClearDepth: Enhanced Stereo Perception of Transparent Objects for Robotic Manipulation](https://mlsanity.com/paper/2409.08926/) (2024)
- [Monocular Depth Estimation and Segmentation for Transparent Object with Iterative Semantic and Geometric Fusion](https://mlsanity.com/paper/2502.14616/) (2025)
- [DistillGrasp: Integrating Features Correlation with Knowledge Distillation for Depth Completion of Transparent Objects](https://mlsanity.com/paper/2408.00337/) (2024)
- [Depth Restoration of Hand-Held Transparent Objects for Human-to-Robot Handover](https://mlsanity.com/paper/2408.14997/) (2024)
- [Self-Supervised Learning for Transparent Object Depth Completion Using Depth from Non-Transparent Objects](https://mlsanity.com/paper/2512.05006/) (2025)
- [AnyOKP: One-Shot and Instance-Aware Object Keypoint Extraction with Pretrained ViT](https://mlsanity.com/paper/2309.08134/) (2023)
- [GAA-TSO: Geometry-Aware Assisted Depth Completion for Transparent and Specular Objects](https://mlsanity.com/paper/2503.17106/) (2025)
- [HTMNet: A Hybrid Network with Transformer-Mamba Bottleneck Multimodal Fusion for Transparent and Reflective Objects Depth Completion](https://mlsanity.com/paper/2505.20904/) (2025)
- [DCIRNet: Depth Completion with Iterative Refinement for Dexterous Grasping of Transparent and Reflective Objects](https://mlsanity.com/paper/2506.09491/) (2025)
- [DidSee: Diffusion-Based Depth Completion for Material-Agnostic Robotic Perception and Manipulation](https://mlsanity.com/paper/2506.21034/) (2025)
- [Rethinking Transparent Object Grasping: Depth Completion with Monocular Depth Estimation and Instance Mask](https://mlsanity.com/paper/2508.02507/) (2025)
- [AISPO: Enhancing Depth Reliability for Robotic Manipulation of Non-Lambertian Objects via Affine-Invariant Shape Prior](https://mlsanity.com/paper/2606.25503/) (2026)
- [One Scene, Two Depths: Probing Geometric Ambiguity in Monocular Foundation Models](https://mlsanity.com/paper/2606.29600/) (2026)

## Abstract (arXiv)

Transparent objects are widely used in industrial automation and daily life. However, robust visual recognition and perception of transparent objects have always been a major challenge. Currently, most commercial-grade depth cameras are still not good at sensing the surfaces of transparent objects due to the refraction and reflection of light. In this work, we present a transformer-based transparent object depth estimation approach from a single RGB-D input. We observe that the global characteristics of the transformer make it easier to extract contextual information to perform depth estimation of transparent areas. In addition, to better enhance the fine-grained features, a feature fusion module (FFM) is designed to assist coherent prediction. Our empirical evidence demonstrates that our model delivers significant improvements in recent popular datasets, e.g., 25% gain on RMSE and 21% gain on REL compared to previous state-of-the-art convolutional-based counterparts in ClearGrasp dataset. Extensive results show that our transformer-based model enables better aggregation of the object's RGB and inaccurate depth information to obtain a better depth representation. Our code and the pre-trained model will be available at https://github.com/yuchendoudou/TODE.