# Foreground-aware Image Inpainting

arXiv: https://arxiv.org/abs/1901.05945 · HTML: https://mlsanity.com/paper/1901.05945/

## Key facts

- Cited by 90 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Jan 2019.

## Built on by (most cited first, 30 of 90)

- [RePaint: Inpainting using Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2201.09865/) (2022)
- [Resolution-robust Large Mask Inpainting with Fourier Convolutions](https://mlsanity.com/paper/2109.07161/) (2021)
- [Large Scale Image Completion via Co-Modulated Generative Adversarial Networks](https://mlsanity.com/paper/2103.10428/) (2021)
- [MAT: Mask-Aware Transformer for Large Hole Image Inpainting](https://mlsanity.com/paper/2203.15270/) (2022)
- [Generating Diverse Structure for Image Inpainting With Hierarchical VQ-VAE](https://mlsanity.com/paper/2103.10022/) (2021)
- [Aggregated Contextual Transformations for High-Resolution Image Inpainting](https://mlsanity.com/paper/2104.01431/) (2021)
- [PD-GAN: Probabilistic Diverse GAN for Image Inpainting](https://mlsanity.com/paper/2105.02201/) (2021)
- [Image Inpainting via Conditional Texture and Structure Dual Generation](https://mlsanity.com/paper/2108.09760/) (2021)
- [Uni-paint: A Unified Framework for Multimodal Image Inpainting with Pretrained Diffusion Model](https://mlsanity.com/paper/2310.07222/) (2023)
- [CM-GAN: Image Inpainting with Cascaded Modulation GAN and Object-Aware Training](https://mlsanity.com/paper/2203.11947/) (2022)
- [TransFill: Reference-guided Image Inpainting by Merging Multiple Color and Spatial Transformations](https://mlsanity.com/paper/2103.15982/) (2021)
- [Deep Learning-based Image and Video Inpainting: A Survey](https://mlsanity.com/paper/2401.03395/) (2024)
- [Image Inpainting by End-to-End Cascaded Refinement with Mask Awareness](https://mlsanity.com/paper/2104.13743/) (2021)
- [SketchEdit: Mask-Free Local Image Manipulation with Partial Sketches](https://mlsanity.com/paper/2111.15078/) (2021)
- [DiffusionEdge: Diffusion Probabilistic Model for Crisp Edge Detection](https://mlsanity.com/paper/2401.02032/) (2024)
- [Learning a Sketch Tensor Space for Image Inpainting of Man-made Scenes](https://mlsanity.com/paper/2103.15087/) (2021)
- [Keys to Better Image Inpainting: Structure and Texture Go Hand in Hand](https://mlsanity.com/paper/2208.03382/) (2022)
- [Image Inpainting with External-internal Learning and Monochromic Bottleneck](https://mlsanity.com/paper/2104.09068/) (2021)
- [Look Ma, No Hands! Agent-Environment Factorization of Egocentric Videos](https://mlsanity.com/paper/2305.16301/) (2023)
- [AutoLink: Self-supervised Learning of Human Skeletons and Object Outlines by Linking Keypoints](https://mlsanity.com/paper/2205.10636/) (2022)
- [Gradpaint: Gradient-Guided Inpainting with Diffusion Models](https://mlsanity.com/paper/2309.09614/) (2023)
- [Layout-Guided Novel View Synthesis from a Single Indoor Panorama](https://mlsanity.com/paper/2103.17022/) (2021)
- [Neural Spline Fields for Burst Image Fusion and Layer Separation](https://mlsanity.com/paper/2312.14235/) (2023)
- [TrajeVAE: Controllable Human Motion Generation from Trajectories](https://mlsanity.com/paper/2104.00351/) (2021)
- [Learning Image-Adaptive Codebooks for Class-Agnostic Image Restoration](https://mlsanity.com/paper/2306.06513/) (2023)
- [Delving Globally into Texture and Structure for Image Inpainting](https://mlsanity.com/paper/2209.08217/) (2022)
- [RGI: robust GAN-inversion for mask-free image inpainting and unsupervised pixel-wise anomaly detection](https://mlsanity.com/paper/2302.12464/) (2023)
- [Deep Video Inpainting Detection](https://mlsanity.com/paper/2101.11080/) (2021)
- [Scene Graph Expansion for Semantics-Guided Image Outpainting](https://mlsanity.com/paper/2205.02958/) (2022)
- [Delving into Crispness: Guided Label Refinement for Crisp Edge Detection](https://mlsanity.com/paper/2306.15172/) (2023)

## Abstract (arXiv)

Existing image inpainting methods typically fill holes by borrowing information from surrounding pixels. They often produce unsatisfactory results when the holes overlap with or touch foreground objects due to lack of information about the actual extent of foreground and background regions within the holes. These scenarios, however, are very important in practice, especially for applications such as the removal of distracting objects. To address the problem, we propose a foreground-aware image inpainting system that explicitly disentangles structure inference and content completion. Specifically, our model learns to predict the foreground contour first, and then inpaints the missing region using the predicted contour as guidance. We show that by such disentanglement, the contour completion model predicts reasonable contours of objects, and further substantially improves the performance of image inpainting. Experiments show that our method significantly outperforms existing methods and achieves superior inpainting results on challenging cases with complex compositions.