# RGB-D Image Inpainting Using Generative Adversarial Network with a Late Fusion Approach

arXiv: https://arxiv.org/abs/2110.07413 · HTML: https://mlsanity.com/paper/2110.07413/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 18 references that are in the map.
- First posted on arXiv 14 Oct 2021.

## Builds on (most cited first, 7 of 7)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Improved Training of Wasserstein GANs](https://mlsanity.com/paper/1704.00028/) (2017)
- [Context Encoders: Feature Learning by Inpainting](https://mlsanity.com/paper/1604.07379/) (2016)
- [Depth Map Prediction from a Single Image using a Multi-Scale Deep Network](https://mlsanity.com/paper/1406.2283/) (2014)
- [Generative Image Inpainting with Contextual Attention](https://mlsanity.com/paper/1801.07892/) (2018)
- [DeepLiDAR: Deep Surface Normal Guided Depth Prediction for Outdoor Scene from Sparse LiDAR Data and Single Color Image](https://mlsanity.com/paper/1812.00488/) (2018)
- [Peeking Behind Objects: Layered Depth Prediction from a Single Image](https://mlsanity.com/paper/1807.08776/) (2018)

## Built on by (most cited first, 3 of 3)

- [Removing Objects From Neural Radiance Fields](https://mlsanity.com/paper/2212.11966/) (2022)
- [InFusion: Inpainting 3D Gaussians via Learning Depth Completion from Diffusion Prior](https://mlsanity.com/paper/2404.11613/) (2024)
- [DeepDR: Deep Structure-Aware RGB-D Inpainting for Diminished Reality](https://mlsanity.com/paper/2312.00532/) (2023)

## Abstract (arXiv)

Diminished reality is a technology that aims to remove objects from video images and fills in the missing region with plausible pixels. Most conventional methods utilize the different cameras that capture the same scene from different viewpoints to allow regions to be removed and restored. In this paper, we propose an RGB-D image inpainting method using generative adversarial network, which does not require multiple cameras. Recently, an RGB image inpainting method has achieved outstanding results by employing a generative adversarial network. However, RGB inpainting methods aim to restore only the texture of the missing region and, therefore, does not recover geometric information (i.e, 3D structure of the scene). We expand conventional image inpainting method to RGB-D image inpainting to jointly restore the texture and geometry of missing regions from a pair of RGB and depth images. Inspired by other tasks that use RGB and depth images (e.g., semantic segmentation and object detection), we propose late fusion approach that exploits the advantage of RGB and depth information each other. The experimental results verify the effectiveness of our proposed method.