# Image Inpainting Guided by Coherence Priors of Semantics and Textures

arXiv: https://arxiv.org/abs/2012.08054 · HTML: https://mlsanity.com/paper/2012.08054/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Dec 2020.

## Built on by (most cited first, 30 of 37)

- [Flow-Guided Transformer for Video Inpainting](https://mlsanity.com/paper/2208.06768/) (2022)
- [Deep Learning-based Image and Video Inpainting: A Survey](https://mlsanity.com/paper/2401.03395/) (2024)
- [Bridging Global Context Interactions for High-Fidelity Image Completion](https://mlsanity.com/paper/2104.00845/) (2021)
- [CoordFill: Efficient High-Resolution Image Inpainting via Parameterized Coordinate Querying](https://mlsanity.com/paper/2303.08524/) (2023)
- [SketchEdit: Mask-Free Local Image Manipulation with Partial Sketches](https://mlsanity.com/paper/2111.15078/) (2021)
- [BIPS: Bi-modal Indoor Panorama Synthesis via Residual Depth-aided Adversarial Learning](https://mlsanity.com/paper/2112.06179/) (2021)
- [Exploring the Effectiveness of Video Perceptual Representation in Blind Video Quality Assessment](https://mlsanity.com/paper/2207.03723/) (2022)
- [High-Fidelity Image Inpainting with GAN Inversion](https://mlsanity.com/paper/2208.11850/) (2022)
- [Delving Globally into Texture and Structure for Image Inpainting](https://mlsanity.com/paper/2209.08217/) (2022)
- [Unbiased Multi-Modality Guidance for Image Inpainting](https://mlsanity.com/paper/2208.11844/) (2022)
- [Boosting Night-time Scene Parsing with Learnable Frequency](https://mlsanity.com/paper/2208.14241/) (2022)
- [Shape-guided Object Inpainting](https://mlsanity.com/paper/2204.07845/) (2022)
- [T-former: An Efficient Transformer for Image Inpainting](https://mlsanity.com/paper/2305.07239/) (2023)
- [Exploiting Optical Flow Guidance for Transformer-Based Video Inpainting](https://mlsanity.com/paper/2301.10048/) (2023)
- [High-Quality Pluralistic Image Completion via Code Shared VQGAN](https://mlsanity.com/paper/2204.01931/) (2022)
- [Reference-Guided Texture and Structure Inference for Image Inpainting](https://mlsanity.com/paper/2207.14498/) (2022)
- [TransRef: Multi-Scale Reference Embedding Transformer for Reference-Guided Image Inpainting](https://mlsanity.com/paper/2306.11528/) (2023)
- [JDSR-GAN: Constructing An Efficient Joint Learning Network for Masked Face Super-Resolution](https://mlsanity.com/paper/2103.13676/) (2021)
- [Towards Interactive Image Inpainting via Sketch Refinement](https://mlsanity.com/paper/2306.00407/) (2023)
- [SuperInpaint: Learning Detail-Enhanced Attentional Implicit Representation for Super-resolutional Image Inpainting](https://mlsanity.com/paper/2307.14489/) (2023)
- [The Challenges of Image Generation Models in Generating Multi-Component Images](https://mlsanity.com/paper/2311.13620/) (2023)
- [DeepDR: Deep Structure-Aware RGB-D Inpainting for Diminished Reality](https://mlsanity.com/paper/2312.00532/) (2023)
- [Interactive Image Inpainting Using Semantic Guidance](https://mlsanity.com/paper/2201.10753/) (2022)
- [Multi-feature Co-learning for Image Inpainting](https://mlsanity.com/paper/2205.10578/) (2022)
- [A Feasibility Study on Image Inpainting for Non-cleft Lip Generation from Patients with Cleft Lip](https://mlsanity.com/paper/2208.01149/) (2022)
- [From Missing Pieces to Masterpieces: Image Completion with Context-Adaptive Diffusion](https://mlsanity.com/paper/2504.14294/) (2025)
- [Visual-aware Attention Dual-stream Decoder for Video Captioning](https://mlsanity.com/paper/2110.08578/) (2021)
- [Photorealistic Facial Wrinkles Removal](https://mlsanity.com/paper/2211.01930/) (2022)
- [Line Drawing Guided Progressive Inpainting of Mural Damage](https://mlsanity.com/paper/2211.06649/) (2022)
- [A Structure-Guided Diffusion Model for Large-Hole Image Completion](https://mlsanity.com/paper/2211.10437/) (2022)

## Abstract (arXiv)

Existing inpainting methods have achieved promising performance in recovering defected images of specific scenes. However, filling holes involving multiple semantic categories remains challenging due to the obscure semantic boundaries and the mixture of different semantic textures. In this paper, we introduce coherence priors between the semantics and textures which make it possible to concentrate on completing separate textures in a semantic-wise manner. Specifically, we adopt a multi-scale joint optimization framework to first model the coherence priors and then accordingly interleavingly optimize image inpainting and semantic segmentation in a coarse-to-fine manner. A Semantic-Wise Attention Propagation (SWAP) module is devised to refine completed image textures across scales by exploring non-local semantic coherence, which effectively mitigates mix-up of textures. We also propose two coherence losses to constrain the consistency between the semantics and the inpainted image in terms of the overall structure and detailed textures. Experimental results demonstrate the superiority of our proposed method for challenging cases with complex holes.