# Diffusion-Based Depth Inpainting for Transparent and Reflective Objects

arXiv: https://arxiv.org/abs/2410.08567 · HTML: https://mlsanity.com/paper/2410.08567/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 46 references that are in the map.
- First posted on arXiv 11 Oct 2024.

## Builds on (most cited first, 30 of 33)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021)
- [Taming Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2012.09841/) (2020)
- [Vision Transformers for Dense Prediction](https://mlsanity.com/paper/2103.13413/) (2021)
- [Perceiver IO: A General Architecture for Structured Inputs & Outputs](https://mlsanity.com/paper/2107.14795/) (2021)
- [NeRDi: Single-View NeRF Synthesis with Language-Guided Diffusion as General Image Priors](https://mlsanity.com/paper/2212.03267/) (2022)
- [High Quality Monocular Depth Estimation via Transfer Learning](https://mlsanity.com/paper/1812.11941/) (2018)
- [ClearGrasp: 3D Shape Estimation of Transparent Objects for Manipulation](https://mlsanity.com/paper/1910.02550/) (2019)
- [Deep Depth Completion of a Single RGB-D Image](https://mlsanity.com/paper/1803.09326/) (2018)
- [Monocular Depth Estimation using Diffusion Models](https://mlsanity.com/paper/2302.14816/) (2023)
- [DiffusionDepth: Diffusion Denoising Approach for Monocular Depth Estimation](https://mlsanity.com/paper/2303.05021/) (2023)
- [iDisc: Internal Discretization for Monocular Depth Estimation](https://mlsanity.com/paper/2304.06334/) (2023)
- [RGB-D Local Implicit Function for Depth Completion of Transparent Objects](https://mlsanity.com/paper/2104.00622/) (2021)
- [Consistent3D: Towards Consistent High-Fidelity Text-to-3D Generation with Deterministic Sampling Prior](https://mlsanity.com/paper/2401.09050/) (2024)
- [Gamba: Marry Gaussian Splatting with Mamba for single view 3D reconstruction](https://mlsanity.com/paper/2403.18795/) (2024)
- [TransCG: A Large-Scale Real-World Dataset for Transparent Object Depth Completion and a Grasping Baseline](https://mlsanity.com/paper/2202.08471/) (2022)
- [Domain Randomization-Enhanced Depth Simulation and Restoration for Perceiving and Grasping Specular and Transparent Objects](https://mlsanity.com/paper/2208.03792/) (2022)
- [Seeing Glass: Joint Point Cloud and Depth Completion for Transparent Objects](https://mlsanity.com/paper/2110.00087/) (2021)
- [Learning Depth Estimation for Transparent and Mirror Surfaces](https://mlsanity.com/paper/2307.15052/) (2023)
- [Robotic Perception of Transparent Objects: A Review](https://mlsanity.com/paper/2304.00157/) (2023)
- [Looking Through the Glass: Neural Surface Reconstruction Against High Specular Reflections](https://mlsanity.com/paper/2304.08706/) (2023)
- [Mirror3D: Depth Refinement for Mirror Surfaces](https://mlsanity.com/paper/2106.06629/) (2021)
- [FDCT: Fast Depth Completion for Transparent Objects](https://mlsanity.com/paper/2307.12274/) (2023)
- [Self-Supervised Monocular Depth Estimation with Self-Reference Distillation and Disparity Offset Refinement](https://mlsanity.com/paper/2302.09789/) (2023)
- [Decomposed Guided Dynamic Filters for Efficient RGB-Guided Depth Completion](https://mlsanity.com/paper/2309.02043/) (2023)
- [Part-Guided 3D RL for Sim2Real Articulated Object Manipulation](https://mlsanity.com/paper/2404.17302/) (2024)
- [Large-Field Contextual Feature Learning for Glass Detection](https://mlsanity.com/paper/2209.04639/) (2022)

## Built on by (most cited first, 9 of 9)

- [Angle Domain Guidance: Latent Diffusion Requires Rotation Rather Than Extrapolation](https://mlsanity.com/paper/2506.11039/) (2025)
- [GeoRemover: Removing Objects and Their Causal Visual Artifacts](https://mlsanity.com/paper/2509.18538/) (2025)
- [Efficient 3D Perception on Multi-Sweep Point Cloud with Gumbel Spatial Pruning](https://mlsanity.com/paper/2411.07742/) (2024)
- [MVAnimate: Enhancing Character Animation with Multi-View Optimization](https://mlsanity.com/paper/2602.08753/) (2026)
- [GAA-TSO: Geometry-Aware Assisted Depth Completion for Transparent and Specular Objects](https://mlsanity.com/paper/2503.17106/) (2025)
- [HTMNet: A Hybrid Network with Transformer-Mamba Bottleneck Multimodal Fusion for Transparent and Reflective Objects Depth Completion](https://mlsanity.com/paper/2505.20904/) (2025)
- [DCIRNet: Depth Completion with Iterative Refinement for Dexterous Grasping of Transparent and Reflective Objects](https://mlsanity.com/paper/2506.09491/) (2025)
- [SeeGroup: Multi-Layer Depth Estimation of Transparent Surfaces via Self-Determined Grouping](https://mlsanity.com/paper/2605.28735/) (2026)
- [RGB-D Video Generation for Improving Human-to-Robot Object Handover Prediction](https://mlsanity.com/paper/2608.13028/) (2026)

## Abstract (arXiv)

Transparent and reflective objects, which are common in our everyday lives, present a significant challenge to 3D imaging techniques due to their unique visual and optical properties. Faced with these types of objects, RGB-D cameras fail to capture the real depth value with their accurate spatial information. To address this issue, we propose DITR, a diffusion-based Depth Inpainting framework specifically designed for Transparent and Reflective objects. This network consists of two stages, including a Region Proposal stage and a Depth Inpainting stage. DITR dynamically analyzes the optical and geometric depth loss and inpaints them automatically. Furthermore, comprehensive experimental results demonstrate that DITR is highly effective in depth inpainting tasks of transparent and reflective objects with robust adaptability.