# Generative Memory-Guided Semantic Reasoning Model for Image Inpainting

arXiv: https://arxiv.org/abs/2110.00261 · HTML: https://mlsanity.com/paper/2110.00261/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 64 references that are in the map.
- First posted on arXiv 1 Oct 2021.

## Builds on (most cited first, 30 of 35)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [Spectral Normalization for Generative Adversarial Networks](https://mlsanity.com/paper/1802.05957/) (2018)
- [Context Encoders: Feature Learning by Inpainting](https://mlsanity.com/paper/1604.07379/) (2016)
- [Instance Normalization: The Missing Ingredient for Fast Stylization](https://mlsanity.com/paper/1607.08022/) (2016)
- [Free-Form Image Inpainting with Gated Convolution](https://mlsanity.com/paper/1806.03589/) (2018)
- [Generative Image Inpainting with Contextual Attention](https://mlsanity.com/paper/1801.07892/) (2018)
- [Image Inpainting for Irregular Holes Using Partial Convolutions](https://mlsanity.com/paper/1804.07723/) (2018)
- [MemNet: A Persistent Memory Network for Image Restoration](https://mlsanity.com/paper/1708.02209/) (2017)
- [Semantic Photo Manipulation with a Generative Image Prior](https://mlsanity.com/paper/2005.07727/) (2020)
- [Pluralistic Image Completion](https://mlsanity.com/paper/1903.04227/) (2019)
- [GLEAN: Generative Latent Bank for Large-Factor Image Super-Resolution](https://mlsanity.com/paper/2012.00739/) (2020)
- [StructureFlow: Image Inpainting via Structure-aware Appearance Flow](https://mlsanity.com/paper/1908.03852/) (2019)
- [Recurrent Feature Reasoning for Image Inpainting](https://mlsanity.com/paper/2008.03737/) (2020)
- [Rethinking Image Inpainting via a Mutual Encoder-Decoder with Feature Equalizations](https://mlsanity.com/paper/2007.06929/) (2020)
- [Coherent Semantic Attention for Image Inpainting](https://mlsanity.com/paper/1905.12384/) (2019)
- [Bringing Old Photos Back to Life](https://mlsanity.com/paper/2004.09484/) (2020)
- [Foreground-aware Image Inpainting](https://mlsanity.com/paper/1901.05945/) (2019)
- [Shift-Net: Image Inpainting via Deep Feature Rearrangement](https://mlsanity.com/paper/1801.09392/) (2018)
- [Generative Face Completion](https://mlsanity.com/paper/1704.05838/) (2017)
- [Image Inpainting via Conditional Texture and Structure Dual Generation](https://mlsanity.com/paper/2108.09760/) (2021)
- [Image Inpainting with Learnable Bidirectional Attention Maps](https://mlsanity.com/paper/1909.00968/) (2019)
- [Single Image Reflection Removal Exploiting Misaligned Training Data and Network Enhancements](https://mlsanity.com/paper/1904.00637/) (2019)
- [Region Normalization for Image Inpainting](https://mlsanity.com/paper/1911.10375/) (2019)

## Built on by (most cited first, 7 of 7)

- [Deep Learning-based Image and Video Inpainting: A Survey](https://mlsanity.com/paper/2401.03395/) (2024)
- [Visual Anomaly Detection Via Partition Memory Bank Module and Error Estimation](https://mlsanity.com/paper/2209.12441/) (2022)
- [Reference-Guided Large-Scale Face Inpainting with Identity and Texture Control](https://mlsanity.com/paper/2303.07014/) (2023)
- [Open Compound Domain Adaptation with Object Style Compensation for Semantic Segmentation](https://mlsanity.com/paper/2309.16127/) (2023)
- [Hierarchical Contrastive Learning for Pattern-Generalizable Image Corruption Detection](https://mlsanity.com/paper/2308.14061/) (2023)
- [Task-Decoupled Image Inpainting Framework for Class-specific Object Remover](https://mlsanity.com/paper/2410.02894/) (2024)
- [Time-variant Image Inpainting via Interactive Distribution Transition Estimation](https://mlsanity.com/paper/2506.23461/) (2025)

## Abstract (arXiv)

Most existing methods for image inpainting focus on learning the intra-image priors from the known regions of the current input image to infer the content of the corrupted regions in the same image. While such methods perform well on images with small corrupted regions, it is challenging for these methods to deal with images with large corrupted area due to two potential limitations: 1) such methods tend to overfit each single training pair of images relying solely on the intra-image prior knowledge learned from the limited known area; 2) the inter-image prior knowledge about the general distribution patterns of visual semantics, which can be transferred across images sharing similar semantics, is not exploited. In this paper, we propose the Generative Memory-Guided Semantic Reasoning Model (GM-SRM), which not only learns the intra-image priors from the known regions, but also distills the inter-image reasoning priors to infer the content of the corrupted regions. In particular, the proposed GM-SRM first pre-learns a generative memory from the whole training data to capture the semantic distribution patterns in a global view. Then the learned memory are leveraged to retrieve the matching inter-image priors for the current corrupted image to perform semantic reasoning during image inpainting. While the intra-image priors are used for guaranteeing the pixel-level content consistency, the inter-image priors are favorable for performing high-level semantic reasoning, which is particularly effective for inferring semantic content for large corrupted area. Extensive experiments on Paris Street View, CelebA-HQ, and Places2 benchmarks demonstrate that our GM-SRM outperforms the state-of-the-art methods for image inpainting in terms of both the visual quality and quantitative metrics.