# Keys to Better Image Inpainting: Structure and Texture Go Hand in Hand

arXiv: https://arxiv.org/abs/2208.03382 · HTML: https://mlsanity.com/paper/2208.03382/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 60 references that are in the map.
- First posted on arXiv 5 Aug 2022.

## Builds on (most cited first, 30 of 34)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [RePaint: Inpainting using Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2201.09865/) (2022)
- [Context Encoders: Feature Learning by Inpainting](https://mlsanity.com/paper/1604.07379/) (2016)
- [Palette: Image-to-Image Diffusion Models](https://mlsanity.com/paper/2111.05826/) (2021)
- [Multi-Scale Context Aggregation by Dilated Convolutions](https://mlsanity.com/paper/1511.07122/) (2015)
- [Semantic Understanding of Scenes through the ADE20K Dataset](https://mlsanity.com/paper/1608.05442/) (2016)
- [Resolution-robust Large Mask Inpainting with Fourier Convolutions](https://mlsanity.com/paper/2109.07161/) (2021)
- [Which Training Methods for GANs do actually Converge?](https://mlsanity.com/paper/1801.04406/) (2018)
- [Free-Form Image Inpainting with Gated Convolution](https://mlsanity.com/paper/1806.03589/) (2018)
- [Generative Image Inpainting with Contextual Attention](https://mlsanity.com/paper/1801.07892/) (2018)
- [Image Inpainting for Irregular Holes Using Partial Convolutions](https://mlsanity.com/paper/1804.07723/) (2018)
- [Dilated Residual Networks](https://mlsanity.com/paper/1705.09914/) (2017)
- [Large Scale Image Completion via Co-Modulated Generative Adversarial Networks](https://mlsanity.com/paper/2103.10428/) (2021)
- [EdgeConnect: Generative Image Inpainting with Adversarial Edge Learning](https://mlsanity.com/paper/1901.00212/) (2019)
- [High-Fidelity Pluralistic Image Completion with Transformers](https://mlsanity.com/paper/2103.14031/) (2021)
- [Contextual Residual Aggregation for Ultra High-Resolution Image Inpainting](https://mlsanity.com/paper/2005.09704/) (2020)
- [Coherent Semantic Attention for Image Inpainting](https://mlsanity.com/paper/1905.12384/) (2019)
- [Foreground-aware Image Inpainting](https://mlsanity.com/paper/1901.05945/) (2019)
- [Aggregated Contextual Transformations for High-Resolution Image Inpainting](https://mlsanity.com/paper/2104.01431/) (2021)
- [High-Resolution Image Inpainting with Iterative Confidence Feedback and Guided Upsampling](https://mlsanity.com/paper/2005.11742/) (2020)
- [Image Inpainting via Conditional Texture and Structure Dual Generation](https://mlsanity.com/paper/2108.09760/) (2021)
- [CM-GAN: Image Inpainting with Cascaded Modulation GAN and Object-Aware Training](https://mlsanity.com/paper/2203.11947/) (2022)
- [TransFill: Reference-guided Image Inpainting by Merging Multiple Color and Spatial Transformations](https://mlsanity.com/paper/2103.15982/) (2021)

## Built on by (most cited first, 25 of 25)

- [SPIn-NeRF: Multiview Segmentation and Perceptual Inpainting with Neural Radiance Fields](https://mlsanity.com/paper/2211.12254/) (2022)
- [Image Restoration Through Generalized Ornstein-Uhlenbeck Bridge](https://mlsanity.com/paper/2312.10299/) (2023)
- [Reference-guided Controllable Inpainting of Neural Radiance Fields](https://mlsanity.com/paper/2304.09677/) (2023)
- [RAD: Region-Aware Diffusion Models for Image Inpainting](https://mlsanity.com/paper/2412.09191/) (2024)
- [PrefPaint: Aligning Image Inpainting Diffusion Model with Human Preference](https://mlsanity.com/paper/2410.21966/) (2024)
- [MaGIC: Multi-modality Guided Image Completion](https://mlsanity.com/paper/2305.11818/) (2023)
- [GeoRemover: Removing Objects and Their Causal Visual Artifacts](https://mlsanity.com/paper/2509.18538/) (2025)
- [Deficiency-Aware Masked Transformer for Video Inpainting](https://mlsanity.com/paper/2307.08629/) (2023)
- [Automatic High Resolution Wire Segmentation and Removal](https://mlsanity.com/paper/2304.00221/) (2023)
- [Reconstructing the Image Stitching Pipeline: Integrating Fusion and Rectangling into a Unified Inpainting Model](https://mlsanity.com/paper/2404.14951/) (2024)
- [Learning Position-Aware Implicit Neural Network for Real-World Face Inpainting](https://mlsanity.com/paper/2401.10537/) (2024)
- [Dense Feature Interaction Network for Image Inpainting Localization](https://mlsanity.com/paper/2408.02191/) (2024)
- [From Missing Pieces to Masterpieces: Image Completion with Context-Adaptive Diffusion](https://mlsanity.com/paper/2504.14294/) (2025)
- [Compensation Sampling for Improved Convergence in Diffusion Models](https://mlsanity.com/paper/2312.06285/) (2023)
- [DMAT: A Dynamic Mask-Aware Transformer for Human De-occlusion](https://mlsanity.com/paper/2402.04558/) (2024)
- [Gorgeous: Create Your Desired Character Facial Makeup from Any Ideas](https://mlsanity.com/paper/2404.13944/) (2024)
- [Inpainting the Gaps: A Novel Framework for Evaluating Explanation Methods in Vision Transformers](https://mlsanity.com/paper/2406.11534/) (2024)
- [Distance Weighted Trans Network for Image Completion](https://mlsanity.com/paper/2310.07440/) (2023)
- [3D-TexSeg: Unsupervised Segmentation of 3D Texture using Mutual Transformer Learning](https://mlsanity.com/paper/2311.10651/) (2023)
- [Windowed-FourierMixer: Enhancing Clutter-Free Room Modeling with Fourier Transform](https://mlsanity.com/paper/2402.18287/) (2024)
- [FaithFill: Faithful Inpainting for Object Completion Using a Single Reference Image](https://mlsanity.com/paper/2406.07865/) (2024)
- [Deep Learning for Sea Surface Temperature Reconstruction under Cloud Occlusion](https://mlsanity.com/paper/2412.03413/) (2024)
- [MILD: Multi-Layer Diffusion Strategy for Complex and Precise Multi-IP Aware Human Erasing](https://mlsanity.com/paper/2508.06543/) (2025)
- [Hidden-Shot: Towards One-Shot Task Generalization for Low-Level Vision Generalist Models](https://mlsanity.com/paper/2607.01535/) (2026)
- [GIFTBench: Diagnosing Generalization in Image Forgery Localization and Informing Model Design](https://mlsanity.com/paper/2610.01778/) (2026)

## Abstract (arXiv)

Deep image inpainting has made impressive progress with recent advances in image generation and processing algorithms. We claim that the performance of inpainting algorithms can be better judged by the generated structures and textures. Structures refer to the generated object boundary or novel geometric structures within the hole, while texture refers to high-frequency details, especially man-made repeating patterns filled inside the structural regions. We believe that better structures are usually obtained from a coarse-to-fine GAN-based generator network while repeating patterns nowadays can be better modeled using state-of-the-art high-frequency fast fourier convolutional layers. In this paper, we propose a novel inpainting network combining the advantages of the two designs. Therefore, our model achieves a remarkable visual quality to match state-of-the-art performance in both structure generation and repeating texture synthesis using a single network. Extensive experiments demonstrate the effectiveness of the method, and our conclusions further highlight the two critical factors of image inpainting quality, structures, and textures, as the future design directions of inpainting networks.