# Context-Aware Image Inpainting with Learned Semantic Priors

arXiv: https://arxiv.org/abs/2106.07220 · HTML: https://mlsanity.com/paper/2106.07220/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 36 references that are in the map.
- First posted on arXiv 14 Jun 2021.

## Builds on (most cited first, 15 of 15)

- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Context Encoders: Feature Learning by Inpainting](https://mlsanity.com/paper/1604.07379/) (2016)
- [Semantic Image Synthesis with Spatially-Adaptive Normalization](https://mlsanity.com/paper/1903.07291/) (2019)
- [Free-Form Image Inpainting with Gated Convolution](https://mlsanity.com/paper/1806.03589/) (2018)
- [Generative Image Inpainting with Contextual Attention](https://mlsanity.com/paper/1801.07892/) (2018)
- [Image Inpainting for Irregular Holes Using Partial Convolutions](https://mlsanity.com/paper/1804.07723/) (2018)
- [Asymmetric Loss For Multi-Label Classification](https://mlsanity.com/paper/2009.14119/) (2020)
- [EdgeConnect: Generative Image Inpainting with Adversarial Edge Learning](https://mlsanity.com/paper/1901.00212/) (2019)
- [Pluralistic Image Completion](https://mlsanity.com/paper/1903.04227/) (2019)
- [StructureFlow: Image Inpainting via Structure-aware Appearance Flow](https://mlsanity.com/paper/1908.03852/) (2019)
- [Recurrent Feature Reasoning for Image Inpainting](https://mlsanity.com/paper/2008.03737/) (2020)
- [Rethinking Image Inpainting via a Mutual Encoder-Decoder with Feature Equalizations](https://mlsanity.com/paper/2007.06929/) (2020)
- [Image Inpainting with Learnable Bidirectional Attention Maps](https://mlsanity.com/paper/1909.00968/) (2019)
- [Region Normalization for Image Inpainting](https://mlsanity.com/paper/1911.10375/) (2019)
- [Guidance and Evaluation: Semantic-Aware Image Inpainting for Mixed Scenes](https://mlsanity.com/paper/2003.06877/) (2020)

## Built on by (most cited first, 9 of 9)

- [Deep Learning-based Image and Video Inpainting: A Survey](https://mlsanity.com/paper/2401.03395/) (2024)
- [ZITS++: Image Inpainting by Improving the Incremental Transformer on Structural Priors](https://mlsanity.com/paper/2210.05950/) (2022)
- [Don't Forget Me: Accurate Background Recovery for Text Removal via Modeling Local-Global Context](https://mlsanity.com/paper/2207.10273/) (2022)
- [A Generalist FaceX via Learning Unified Facial Representation](https://mlsanity.com/paper/2401.00551/) (2023)
- [Coarse-to-fine Task-driven Inpainting for Geoscience Images](https://mlsanity.com/paper/2211.11059/) (2022)
- [Fully Context-Aware Image Inpainting with a Learned Semantic Pyramid](https://mlsanity.com/paper/2112.04107/) (2021)
- [Take a Prior from Other Tasks for Severe Blur Removal](https://mlsanity.com/paper/2302.06898/) (2023)
- [Leveraging Text Localization for Scene Text Removal via Text-aware Masked Image Modeling](https://mlsanity.com/paper/2409.13431/) (2024)
- [Multi-scale Semantic Prior Features Guided Deep Neural Network for Urban Street-view Image](https://mlsanity.com/paper/2405.10504/) (2024)

## Abstract (arXiv)

Recent advances in image inpainting have shown impressive results for generating plausible visual details on rather simple backgrounds. However, for complex scenes, it is still challenging to restore reasonable contents as the contextual information within the missing regions tends to be ambiguous. To tackle this problem, we introduce pretext tasks that are semantically meaningful to estimating the missing contents. In particular, we perform knowledge distillation on pretext models and adapt the features to image inpainting. The learned semantic priors ought to be partially invariant between the high-level pretext task and low-level image inpainting, which not only help to understand the global context but also provide structural guidance for the restoration of local textures. Based on the semantic priors, we further propose a context-aware image inpainting model, which adaptively integrates global semantics and local features in a unified image generator. The semantic learner and the image generator are trained in an end-to-end manner. We name the model SPL to highlight its ability to learn and leverage semantic priors. It achieves the state of the art on Places2, CelebA, and Paris StreetView datasets.