mlsanityData to 7 Oct 2026

RGB-D Local Implicit Function for Depth Completion of Transparent Objects

arXiv 2104.00622 ↗First posted 1 Apr 2021cs.CV cs.ROcited by 64

Not on the map. Why?

Abstract (arXiv)

Majority of the perception methods in robotics require depth information provided by RGB-D cameras. However, standard 3D sensors fail to capture depth of transparent objects due to refraction and absorption of light. In this paper, we introduce a new approach for depth completion of transparent objects from a single RGB-D image. Key to our approach is a local implicit neural representation built on ray-voxel pairs that allows our method to generalize to unseen objects and achieve fast inference speed. Based on this representation, we present a novel framework that can complete missing depth given noisy RGB-D input. We further improve the depth estimation iteratively using a self-correcting refinement model. To train the whole pipeline, we build a large scale synthetic dataset with transparent objects. Experiments demonstrate that our method performs significantly better than the current state-of-the-art methods on both synthetic and real world data. In addition, our approach improves the inference speed by a factor of 20 compared to the previous best method, ClearGrasp. Code and dataset will be released at https://research.nvidia.com/publication/2021-03_RGB-D-Local-Implicit.

Key facts. Cited by 64 papers in the map (data through 7 Oct 2026). Builds on 38 of its 58 references that are in the map. First posted on arXiv 1 Apr 2021.

Builds on

38 of 58 references.

References · 38

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2017Mask R-CNNcited by 7,259
  3. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  4. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
  5. 2013Describing Textures in the Wildcited by 2,110
  6. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,037
  7. 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,484
  8. 2014Depth Map Prediction from a Single Image using a Multi-Scale Deep Networkcited by 1,264
  9. 2018Learning Implicit Fields for Generative Shape Modelingcited by 1,060
  10. 2020Neural Sparse Voxel Fieldscited by 959
  11. 2016Unsupervised Monocular Depth Estimation with Left-Right Consistencycited by 819
  12. 2019PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitizationcited by 818
26 more
  1. 2020Convolutional Occupancy Networkscited by 760
  2. 2018Deep Ordinal Regression Network for Monocular Depth Estimationcited by 721
  3. 2020GRAF: Generative Radiance Fields for 3D-Aware Image Synthesiscited by 688
  4. 2016Deeper Depth Prediction with Fully Convolutional Residual Networkscited by 402
  5. 2018PCN: Point Completion Networkcited by 382
  6. 2020Local Implicit Grid Representations for 3D Scenescited by 372
  7. 2016Unsupervised CNN for Single View Depth Estimation: Geometry to the Rescuecited by 368
  8. 2020Implicit Functions in Feature Space for 3D Shape Reconstruction and Completioncited by 343
  9. 2019DISN: Deep Implicit Surface Network for High-quality Single-view 3D Reconstructioncited by 332
  10. 2017Sparsity Invariant CNNscited by 325
  11. 2020Deep Local Shapes: Learning Local SDF Priors for Detailed 3D Reconstructioncited by 321
  12. 2019Learning Shape Templates with Structured Implicit Functionscited by 242
  13. 2020Non-Local Spatial Propagation Network for Depth Completioncited by 194
  14. 2016Single-Image Depth Perception in the Wildcited by 179
  15. 2017Sparse-to-Dense: Depth Prediction from Sparse Depth Samples and a Single Imagecited by 159
  16. 2018DeepLiDAR: Deep Surface Normal Guided Depth Prediction for Outdoor Scene from Sparse LiDAR Data and Single Color Imagecited by 129
  17. 2019ClearGrasp: 3D Shape Estimation of Transparent Objects for Manipulationcited by 124
  18. 2015Human Pose Estimation with Iterative Error Feedbackcited by 121
  19. 2018Deep Depth Completion of a Single RGB-D Imagecited by 110
  20. 2018Depth Estimation via Affinity Learned with Convolutional Spatial Propagation Networkcited by 109
  21. 2019Depth Completion from Sparse LiDAR Data with Depth-Normal Constraintscited by 108
  22. 2019KeyPose: Multi-View 3D Labeling and Keypoint Estimation for Transparent Objectscited by 75
  23. 2020Learning Joint 2D-3D Representations for Depth Completioncited by 71
  24. 2020Learning RGB-D Feature Embeddings for Unseen Object Instance Segmentationcited by 64
  25. 2020Through the Looking Glass: Neural 3D Reconstruction of Transparent Shapescited by 32
  26. 2018Full 3D Reconstruction of Transparent Objectscited by 24

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 5; 2022: 11; 2023: 11; 2024: 10; 2025: 17; 2026: 10.

64 papers.

Sort:

By area:

  1. 2021Neural Fields in Visual Computing and Beyondcited by 539
  2. 2021Dex-NeRF: Using a Neural Radiance Field to Grasp Transparent Objectscited by 119
  3. 2022Conditional-Flow NeRF: Accurate 3D Modelling with Reliable Uncertainty Quantificationcited by 71
  4. 2022TransCG: A Large-Scale Real-World Dataset for Transparent Object Depth Completion and a Grasping Baselinecited by 55
  5. 2022Domain Randomization-Enhanced Depth Simulation and Restoration for Perceiving and Grasping Specular and Transparent Objectscited by 49
  6. 2022ClearPose: Large-scale Transparent Object Dataset and Benchmarkcited by 46
  7. 2021Toward Practical Monocular Indoor Depth Estimationcited by 42
  8. 2021Seeing Glass: Joint Point Cloud and Depth Completion for Transparent Objectscited by 39
  9. 2023D3Fields: Dynamic 3D Descriptor Fields for Zero-Shot Generalizable Rearrangementcited by 35
  10. 2022StereoPose: Category-Level 6D Transparent Object Pose Estimation from Stereo Images via Back-View NOCScited by 18
  11. 2022Transparent Shape from a Single View Polarization Imagecited by 16
  12. 2023MVTrans: Multi-View Perception of Transparent Objectscited by 14
38 more
  1. 2022TransNet: Category-Level Transparent Object Pose Estimationcited by 13
  2. 2022TODE-Trans: Transparent Object Depth Estimation with Transformercited by 13
  3. 2023FDCT: Fast Depth Completion for Transparent Objectscited by 12
  4. 2024LayeredFlow: A Real-World Benchmark for Non-Lambertian Multi-Layer Optical Flowcited by 11
  5. 2025IAAO: Interactive Affordance Learning for Articulated Objects in 3D Environmentscited by 11
  6. 2022Self-supervised Transparent Liquid Segmentation for Robotic Pouringcited by 9
  7. 2024ASGrasp: Generalizable Transparent Object Reconstruction and 6-DoF Grasp Detection from RGB-D Active Stereo Cameracited by 9
  8. 2024Diffusion-Based Depth Inpainting for Transparent and Reflective Objectscited by 9
  9. 2025TranSplat: Surface Embedding-guided 3D Gaussian Splatting for Transparent Object Manipulationcited by 9
  10. 2025Monocular Depth Estimation and Segmentation for Transparent Object with Iterative Semantic and Geometric Fusioncited by 8
  11. 2025Seeing and Seeing Through the Glass: Real and Synthetic Data for Multi-Layer Depth Estimationcited by 8
  12. 2024ClearDepth: Enhanced Stereo Perception of Transparent Objects for Robotic Manipulationcited by 7
  13. 2025TransDiff: Diffusion-Based Method for Manipulating Transparent Objects Using a Single RGB-D Imagecited by 7
  14. 2023SA6D: Self-Adaptive Few-Shot 6D Pose Estimator for Novel and Occluded Objectscited by 6
  15. 2023RFTrans: Leveraging Refractive Flow of Transparent Objects for Surface Normal Estimation and Manipulationcited by 6
  16. 2021SuperCaustics: Real-time, open-source simulation of transparent objects for deep learning applicationscited by 5
  17. 2022Visual-tactile Fusion for Transparent Object Grasping in Complex Backgroundscited by 5
  18. 2023Virtual Occlusions Through Implicit Depthcited by 5
  19. 2024RaSim: A Range-aware High-fidelity RGB-D Data Simulation Pipeline for Real-world Applicationscited by 5
  20. 2025DepthFocus: Controllable Depth Estimation for See-Through Scenescited by 3
  21. 2023TransTouch: Learning Transparent Objects Depth Sensing Through Sparse Touchescited by 2
  22. 2023NeuralLabeling: A versatile toolset for labeling vision datasets using Neural Radiance Fieldscited by 2
  23. 2025Monocular One-Shot Metric-Depth Alignment for RGB-Based Robot Graspingcited by 2
  24. 2026TransNormal: Dense Visual Semantics for Diffusion-based Transparent Object Normal Estimationcited by 2
  25. 2022Grasping the Inconspicuouscited by 1
  26. 2023TransNet: Transparent Object Manipulation Through Category-Level Pose Estimationcited by 1
  27. 2024Transparent Object Depth Completioncited by 1
  28. 2024DistillGrasp: Integrating Features Correlation with Knowledge Distillation for Depth Completion of Transparent Objectscited by 1
  29. 2024Depth Restoration of Hand-Held Transparent Objects for Human-to-Robot Handovercited by 1
  30. 2025Shaken, Not Stirred: A Novel Dataset for Visual Understanding of Glasses in Human-Robot Bartending Taskscited by 1
  31. 2025StarryGazer: Leveraging Monocular Depth Estimation Models for Domain-Agnostic Single Depth Image Completioncited by 1
  32. 2026GeomPrompt: Geometric Prompt Learning for RGB-D Semantic Segmentation Under Missing and Degraded Depthcited by 1
  33. 2026AnchorD: Metric Grounding of Monocular Depth Using Factor Graphscited by 1
  34. 2026PhysGraph: A Physics-aware 3D Scene Graph for Perception and Reasoningcited by 1
  35. 2023CN-DHF: Compact Neural Double Height-Field Representations of 3D Shapescited by 0
  36. 2023Tabletop Transparent Scene Reconstruction via Epipolar-Guided Optical Flow with Monocular Depth Completion Priorcited by 0
  37. 2024VAIR: Visuo-Acoustic Implicit Representations for Low-Cost, Multi-Modal Transparent Surface Reconstruction in Indoor Scenescited by 0
  38. 2024ReFlow6D: Refraction-Guided Transparent Object 6D Pose Estimation via Intermediate Representation Learningcited by 0

50 most cited of 64 All 64 (JSON)