# Depth-Aware Endoscopic Video Inpainting

arXiv: https://arxiv.org/abs/2407.02675 · HTML: https://mlsanity.com/paper/2407.02675/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 26 references that are in the map.
- First posted on arXiv 2 Jul 2024.

## Builds on (most cited first, 10 of 10)

- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [ProPainter: Improving Propagation and Transformer for Video Inpainting](https://mlsanity.com/paper/2309.03897/) (2023)
- [Learning Joint Spatial-Temporal Transformations for Video Inpainting](https://mlsanity.com/paper/2007.10247/) (2020)
- [Free-form Video Inpainting with 3D Gated Convolution and Temporal PatchGAN](https://mlsanity.com/paper/1904.10247/) (2019)
- [Self-Supervised Monocular Depth and Ego-Motion Estimation in Endoscopy: Appearance Flow to the Rescue](https://mlsanity.com/paper/2112.08122/) (2021)
- [Video Inpainting of Complex Scenes](https://mlsanity.com/paper/1503.05528/) (2015)
- [A deep learning framework for quality assessment and restoration in video endoscopy](https://mlsanity.com/paper/1904.07073/) (2019)
- [SERV-CT: A disparity dataset from CT for validation of endoscopic 3D reconstruction](https://mlsanity.com/paper/2012.11779/) (2020)
- [DVI: Depth Guided Video Inpainting for Autonomous Driving](https://mlsanity.com/paper/2007.08854/) (2020)

## Built on by (most cited first, 2 of 2)

- [Rethinking Brain Tumor Segmentation from the Frequency Domain Perspective](https://mlsanity.com/paper/2506.10142/) (2025)
- [TRACE: Temporally Reliable Anatomically-Conditioned 3D CT Generation with Enhanced Efficiency](https://mlsanity.com/paper/2507.00802/) (2025)

## Abstract (arXiv)

Video inpainting fills in corrupted video content with plausible replacements. While recent advances in endoscopic video inpainting have shown potential for enhancing the quality of endoscopic videos, they mainly repair 2D visual information without effectively preserving crucial 3D spatial details for clinical reference. Depth-aware inpainting methods attempt to preserve these details by incorporating depth information. Still, in endoscopic contexts, they face challenges including reliance on pre-acquired depth maps, less effective fusion designs, and ignorance of the fidelity of 3D spatial details. To address them, we introduce a novel Depth-aware Endoscopic Video Inpainting (DAEVI) framework. It features a Spatial-Temporal Guided Depth Estimation module for direct depth estimation from visual features, a Bi-Modal Paired Channel Fusion module for effective channel-by-channel fusion of visual and depth information, and a Depth Enhanced Discriminator to assess the fidelity of the RGB-D sequence comprised of the inpainted frames and estimated depth images. Experimental evaluations on established benchmarks demonstrate our framework's superiority, achieving a 2% improvement in PSNR and a 6% reduction in MSE compared to state-of-the-art methods. Qualitative analyses further validate its enhanced ability to inpaint fine details, highlighting the benefits of integrating depth information into endoscopic inpainting.