# RGB-Depth Fusion GAN for Indoor Depth Completion

arXiv: https://arxiv.org/abs/2203.10856 · HTML: https://mlsanity.com/paper/2203.10856/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 48 references that are in the map.
- First posted on arXiv 21 Mar 2022.

## Builds on (most cited first, 24 of 24)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Improved Training of Wasserstein GANs](https://mlsanity.com/paper/1704.00028/) (2017)
- [Conditional Generative Adversarial Nets](https://mlsanity.com/paper/1411.1784/) (2014)
- [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization](https://mlsanity.com/paper/1703.06868/) (2017)
- [Deep Hough Voting for 3D Object Detection in Point Clouds](https://mlsanity.com/paper/1904.09664/) (2019)
- [Non-Local Spatial Propagation Network for Depth Completion](https://mlsanity.com/paper/2007.10042/) (2020)
- [Sparse-to-Dense: Depth Prediction from Sparse Depth Samples and a Single Image](https://mlsanity.com/paper/1709.07492/) (2017)
- [DeepLiDAR: Deep Surface Normal Guided Depth Prediction for Outdoor Scene from Sparse LiDAR Data and Single Color Image](https://mlsanity.com/paper/1812.00488/) (2018)
- [H3DNet: 3D Object Detection Using Hybrid Geometric Primitives](https://mlsanity.com/paper/2006.05682/) (2020)
- [Depth Estimation via Affinity Learned with Convolutional Spatial Propagation Network](https://mlsanity.com/paper/1808.00150/) (2018)
- [Intel RealSense Stereoscopic Depth Cameras](https://mlsanity.com/paper/1705.05548/) (2017)
- [Learning Affinity via Spatial Propagation Networks](https://mlsanity.com/paper/1710.01020/) (2017)
- [Sparse and noisy LiDAR completion with RGB guidance and uncertainty](https://mlsanity.com/paper/1902.05356/) (2019)
- [Depth Coefficients for Depth Completion](https://mlsanity.com/paper/1903.05421/) (2019)
- [Indoor Depth Completion with Boundary Consistency and Self-Attention](https://mlsanity.com/paper/1908.08344/) (2019)
- [Depth Completion using Plane-Residual Representation](https://mlsanity.com/paper/2104.07350/) (2021)
- [Translate-to-Recognize Networks for RGB-D Scene Recognition](https://mlsanity.com/paper/1904.12254/) (2019)
- [PointAR: Efficient Lighting Estimation for Mobile Augmented Reality](https://mlsanity.com/paper/2004.00006/) (2020)
- [Multi-task GANs for Semantic Segmentation and Depth Completion with Cycle Consistency](https://mlsanity.com/paper/2011.14272/) (2020)
- [Affinity CNN: Learning Pixel-Centric Pairwise Relations for Figure/Ground Embedding](https://mlsanity.com/paper/1512.02767/) (2015)

## Built on by (most cited first, 18 of 18)

- [Completion as Enhancement: A Degradation-Aware Selective Image Guided Network for Depth Completion](https://mlsanity.com/paper/2412.19225/) (2024)
- [CP3: Channel Pruning Plug-in for Point-based Networks](https://mlsanity.com/paper/2303.13097/) (2023)
- [RoadFormer+: Delivering RGB-X Scene Parsing through Scale-Aware Information Decoupling and Advanced Heterogeneous Feature Fusion](https://mlsanity.com/paper/2407.21631/) (2024)
- [AGG-Net: Attention Guided Gated-convolutional Network for Depth Image Completion](https://mlsanity.com/paper/2309.01624/) (2023)
- [RGB Guided ToF Imaging System: A Survey of Deep Learning-based Methods](https://mlsanity.com/paper/2405.10357/) (2024)
- [RDFC-GAN: RGB-Depth Fusion CycleGAN for Indoor Depth Completion](https://mlsanity.com/paper/2306.03584/) (2023)
- [DTF-Net: Category-Level Pose Estimation and Shape Reconstruction via Deformable Template Field](https://mlsanity.com/paper/2308.02239/) (2023)
- [Any to Full: Prompting Depth Anything for Depth Completion in One Stage](https://mlsanity.com/paper/2603.05711/) (2026)
- [Decomposed Vector-Quantized Variational Autoencoder for Human Grasp Generation](https://mlsanity.com/paper/2407.14062/) (2024)
- [DeepDR: Deep Structure-Aware RGB-D Inpainting for Diminished Reality](https://mlsanity.com/paper/2312.00532/) (2023)
- [Propagating Sparse Depth via Depth Foundation Model for Out-of-Distribution Depth Completion](https://mlsanity.com/paper/2508.04984/) (2025)
- [Improving Neural Indoor Surface Reconstruction with Mask-Guided Adaptive Consistency Constraints](https://mlsanity.com/paper/2309.09739/) (2023)
- [A VideoMAE-v2 Approach to Zero-Shot Traffic Accident Anticipation](https://mlsanity.com/paper/2606.09542/) (2026)
- [Mask-adaptive Gated Convolution and Bi-directional Progressive Fusion Network for Depth Completion](https://mlsanity.com/paper/2401.07439/) (2024)
- [GAA-TSO: Geometry-Aware Assisted Depth Completion for Transparent and Specular Objects](https://mlsanity.com/paper/2503.17106/) (2025)
- [HTMNet: A Hybrid Network with Transformer-Mamba Bottleneck Multimodal Fusion for Transparent and Reflective Objects Depth Completion](https://mlsanity.com/paper/2505.20904/) (2025)
- [S2ML: Spatio-Spectral Mutual Learning for Depth Completion](https://mlsanity.com/paper/2511.06033/) (2025)
- [Leveraging Metric Depth for Relative Depth Prediction](https://mlsanity.com/paper/2606.10628/) (2026)

## Abstract (arXiv)

The raw depth image captured by the indoor depth sensor usually has an extensive range of missing depth values due to inherent limitations such as the inability to perceive transparent objects and limited distance range. The incomplete depth map burdens many downstream vision tasks, and a rising number of depth completion methods have been proposed to alleviate this issue. While most existing methods can generate accurate dense depth maps from sparse and uniformly sampled depth maps, they are not suitable for complementing the large contiguous regions of missing depth values, which is common and critical. In this paper, we design a novel two-branch end-to-end fusion network, which takes a pair of RGB and incomplete depth images as input to predict a dense and completed depth map. The first branch employs an encoder-decoder structure to regress the local dense depth values from the raw depth map, with the help of local guidance information extracted from the RGB image. In the other branch, we propose an RGB-depth fusion GAN to transfer the RGB image to the fine-grained textured depth map. We adopt adaptive fusion modules named W-AdaIN to propagate the features across the two branches, and we append a confidence fusion head to fuse the two outputs of the branches for the final depth map. Extensive experiments on NYU-Depth V2 and SUN RGB-D demonstrate that our proposed method clearly improves the depth completion performance, especially in a more realistic setting of indoor environments with the help of the pseudo depth map.