# Texture Memory-Augmented Deep Patch-Based Image Inpainting

arXiv: https://arxiv.org/abs/2009.13240 · HTML: https://mlsanity.com/paper/2009.13240/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Sep 2020.

## Built on by (most cited first, 22 of 22)

- [Re-Imagen: Retrieval-Augmented Text-to-Image Generator](https://mlsanity.com/paper/2209.14491/) (2022)
- [Text-Guided Synthesis of Artistic Images with Retrieval-Augmented Diffusion Models](https://mlsanity.com/paper/2207.13038/) (2022)
- [Implicit Identity Representation Conditioned Memory Compensation Network for Talking Head video Generation](https://mlsanity.com/paper/2307.09906/) (2023)
- [Retrieval-Enhanced Machine Learning](https://mlsanity.com/paper/2205.01230/) (2022)
- [Deep Learning-based Image and Video Inpainting: A Survey](https://mlsanity.com/paper/2401.03395/) (2024)
- [Semi-Parametric Neural Image Synthesis](https://mlsanity.com/paper/2204.11824/) (2022)
- [RetrievalFuse: Neural 3D Scene Reconstruction with a Database](https://mlsanity.com/paper/2104.00024/) (2021)
- [Retrieval-Augmented Open-Vocabulary Object Detection](https://mlsanity.com/paper/2404.05687/) (2024)
- [HumanDiffusion: a Coarse-to-Fine Alignment Diffusion Framework for Controllable Text-Driven Person Image Generation](https://mlsanity.com/paper/2211.06235/) (2022)
- [Open Compound Domain Adaptation with Object Style Compensation for Semantic Segmentation](https://mlsanity.com/paper/2309.16127/) (2023)
- [Generative Memory-Guided Semantic Reasoning Model for Image Inpainting](https://mlsanity.com/paper/2110.00261/) (2021)
- [PatchRD: Detail-Preserving Shape Completion by Learning Patch Retrieval and Deformation](https://mlsanity.com/paper/2207.11790/) (2022)
- [Depth-Synergized Mamba Meets Memory Experts for All-Day Image Reflection Separation](https://mlsanity.com/paper/2601.00322/) (2026)
- [Remember What You have drawn: Semantic Image Manipulation with Memory](https://mlsanity.com/paper/2107.12579/) (2021)
- [ShowFace: Coordinated Face Inpainting with Memory-Disentangled Refinement Networks](https://mlsanity.com/paper/2204.02824/) (2022)
- [Learning to Scale Temperature in Masked Self-Attention for Image Inpainting](https://mlsanity.com/paper/2302.06130/) (2023)
- [Learning Position-Aware Implicit Neural Network for Real-World Face Inpainting](https://mlsanity.com/paper/2401.10537/) (2024)
- [LayoutRAG: Retrieval-Augmented Model for Content-agnostic Conditional Layout Generation](https://mlsanity.com/paper/2506.02697/) (2025)
- [Adapting to Skew: Imputing Spatiotemporal Urban Data with 3D Partial Convolutions and Biased Masking](https://mlsanity.com/paper/2301.04233/) (2023)
- [E2F-Net: Eyes-to-Face Inpainting via StyleGAN Latent Space](https://mlsanity.com/paper/2403.12197/) (2024)
- [TrackingMiM: Efficient Mamba-in-Mamba Serialization for Real-time UAV Object Tracking](https://mlsanity.com/paper/2507.01535/) (2025)
- [PatchEX: High-Quality Real-Time Temporal Supersampling through Patch-based Parallel Extrapolation](https://mlsanity.com/paper/2407.17501/) (2024)

## Abstract (arXiv)

Patch-based methods and deep networks have been employed to tackle image inpainting problem, with their own strengths and weaknesses. Patch-based methods are capable of restoring a missing region with high-quality texture through searching nearest neighbor patches from the unmasked regions. However, these methods bring problematic contents when recovering large missing regions. Deep networks, on the other hand, show promising results in completing large regions. Nonetheless, the results often lack faithful and sharp details that resemble the surrounding area. By bringing together the best of both paradigms, we propose a new deep inpainting framework where texture generation is guided by a texture memory of patch samples extracted from unmasked regions. The framework has a novel design that allows texture memory retrieval to be trained end-to-end with the deep inpainting network. In addition, we introduce a patch distribution loss to encourage high-quality patch synthesis. The proposed method shows superior performance both qualitatively and quantitatively on three challenging image benchmarks, i.e., Places, CelebA-HQ, and Paris Street-View datasets.