# A4T: Hierarchical Affordance Detection for Transparent Objects Depth Reconstruction and Manipulation

arXiv: https://arxiv.org/abs/2207.04907 · HTML: https://mlsanity.com/paper/2207.04907/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Jul 2022.

## Built on by (most cited first, 23 of 23)

- [Open-Vocabulary Affordance Detection in 3D Point Clouds](https://mlsanity.com/paper/2303.02401/) (2023)
- [StereoPose: Category-Level 6D Transparent Object Pose Estimation from Stereo Images via Back-View NOCS](https://mlsanity.com/paper/2211.01644/) (2022)
- [TRansPose: Large-Scale Multispectral Dataset for Transparent Object](https://mlsanity.com/paper/2307.05016/) (2023)
- [ClearDepth: Enhanced Stereo Perception of Transparent Objects for Robotic Manipulation](https://mlsanity.com/paper/2409.08926/) (2024)
- [Monocular Depth Estimation and Segmentation for Transparent Object with Iterative Semantic and Geometric Fusion](https://mlsanity.com/paper/2502.14616/) (2025)
- [TransDiff: Diffusion-Based Method for Manipulating Transparent Objects Using a Single RGB-D Image](https://mlsanity.com/paper/2503.12779/) (2025)
- [RFTrans: Leveraging Refractive Flow of Transparent Objects for Surface Normal Estimation and Manipulation](https://mlsanity.com/paper/2311.12398/) (2023)
- [Visual-tactile Fusion for Transparent Object Grasping in Complex Backgrounds](https://mlsanity.com/paper/2211.16693/) (2022)
- [Segmenting Object Affordances: Reproducibility and Sensitivity to Scale](https://mlsanity.com/paper/2409.01814/) (2024)
- [Affordance segmentation of hand-occluded containers from exocentric images](https://mlsanity.com/paper/2308.11233/) (2023)
- [REF2-NeRF: Reflection and Refraction aware Neural Radiance Field](https://mlsanity.com/paper/2311.17116/) (2023)
- [TRAN-D: 2D Gaussian Splatting-based Sparse-view Transparent Object Depth Reconstruction via Physics Simulation for Scene Update](https://mlsanity.com/paper/2507.11069/) (2025)
- [DeLTa: Demonstration and Language-Guided Novel Transparent Object Manipulation](https://mlsanity.com/paper/2510.05662/) (2025)
- [Shaken, Not Stirred: A Novel Dataset for Visual Understanding of Glasses in Human-Robot Bartending Tasks](https://mlsanity.com/paper/2503.04308/) (2025)
- [Visual Affordance Prediction: Survey and Reproducibility](https://mlsanity.com/paper/2505.05074/) (2025)
- [EGSA-PT:Edge-Guided Spatial Attention with Progressive Training for Monocular Depth Estimation and Segmentation of Transparent Objects](https://mlsanity.com/paper/2511.14970/) (2025)
- [Self-Supervised Learning for Transparent Object Depth Completion Using Depth from Non-Transparent Objects](https://mlsanity.com/paper/2512.05006/) (2025)
- [HTMNet: A Hybrid Network with Transformer-Mamba Bottleneck Multimodal Fusion for Transparent and Reflective Objects Depth Completion](https://mlsanity.com/paper/2505.20904/) (2025)
- [Capturing Fine-Grained Alignments Improves 3D Affordance Detection](https://mlsanity.com/paper/2506.19312/) (2025)
- [Rethinking Transparent Object Grasping: Depth Completion with Monocular Depth Estimation and Instance Mask](https://mlsanity.com/paper/2508.02507/) (2025)
- [TransDex: Pre-training Visuo-Tactile Policy with Point Cloud Reconstruction for Dexterous Manipulation of Transparent Objects](https://mlsanity.com/paper/2603.13869/) (2026)
- [GIFT: Geometry-Invariant Fine-Tuning for Non-Lambertian Monocular Depth Estimation](https://mlsanity.com/paper/2608.02068/) (2026)
- [Reproducible Multimodal Affordance Prediction](https://mlsanity.com/paper/2608.18317/) (2026)

## Abstract (arXiv)

Transparent objects are widely used in our daily lives and therefore robots need to be able to handle them. However, transparent objects suffer from light reflection and refraction, which makes it challenging to obtain the accurate depth maps required to perform handling tasks. In this paper, we propose a novel affordance-based framework for depth reconstruction and manipulation of transparent objects, named A4T. A hierarchical AffordanceNet is first used to detect the transparent objects and their associated affordances that encode the relative positions of an object's different parts. Then, given the predicted affordance map, a multi-step depth reconstruction method is used to progressively reconstruct the depth maps of transparent objects. Finally, the reconstructed depth maps are employed for the affordance-based manipulation of transparent objects. To evaluate our proposed method, we construct a real-world dataset TRANS-AFF with affordances and depth maps of transparent objects, which is the first of its kind. Extensive experiments show that our proposed methods can predict accurate affordance maps, and significantly improve the depth reconstruction of transparent objects compared to the state-of-the-art method, with the Root Mean Squared Error in meters significantly decreased from 0.097 to 0.042. Furthermore, we demonstrate the effectiveness of our proposed method with a series of robotic manipulation experiments on transparent objects. See supplementary video and results at https://sites.google.com/view/affordance4trans.