# Delving Globally into Texture and Structure for Image Inpainting arXiv: https://arxiv.org/abs/2209.08217 ยท HTML: https://mlsanity.com/paper/2209.08217/ ## Key facts - Cited by 13 papers in the map (data through 14 Oct 2026). - Builds on 26 of its 45 references that are in the map. - First posted on arXiv 17 Sep 2022. ## Builds on (most cited first, 26 of 26) - [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017) - [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015) - [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020) - [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020) - [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014) - [Context Encoders: Feature Learning by Inpainting](https://mlsanity.com/paper/1604.07379/) (2016) - [Free-Form Image Inpainting with Gated Convolution](https://mlsanity.com/paper/1806.03589/) (2018) - [Generative Image Inpainting with Contextual Attention](https://mlsanity.com/paper/1801.07892/) (2018) - [Image Inpainting for Irregular Holes Using Partial Convolutions](https://mlsanity.com/paper/1804.07723/) (2018) - [Pluralistic Image Completion](https://mlsanity.com/paper/1903.04227/) (2019) - [TransGAN: Two Pure Transformers Can Make One Strong GAN, and That Can Scale Up](https://mlsanity.com/paper/2102.07074/) (2021) - [High-Fidelity Pluralistic Image Completion with Transformers](https://mlsanity.com/paper/2103.14031/) (2021) - [Contextual Residual Aggregation for Ultra High-Resolution Image Inpainting](https://mlsanity.com/paper/2005.09704/) (2020) - [StructureFlow: Image Inpainting via Structure-aware Appearance Flow](https://mlsanity.com/paper/1908.03852/) (2019) - [ViTGAN: Training GANs with Vision Transformers](https://mlsanity.com/paper/2107.04589/) (2021) - [Rethinking Image Inpainting via a Mutual Encoder-Decoder with Feature Equalizations](https://mlsanity.com/paper/2007.06929/) (2020) - [Learning Pyramid-Context Encoder Network for High-Quality Image Inpainting](https://mlsanity.com/paper/1904.07475/) (2019) - [Foreground-aware Image Inpainting](https://mlsanity.com/paper/1901.05945/) (2019) - [Diverse Image Inpainting with Bidirectional and Autoregressive Transformers](https://mlsanity.com/paper/2104.12335/) (2021) - [Image Inpainting via Conditional Texture and Structure Dual Generation](https://mlsanity.com/paper/2108.09760/) (2021) - [Image Inpainting with Learnable Bidirectional Attention Maps](https://mlsanity.com/paper/1909.00968/) (2019) - [Survey on Deep Multi-modal Data Analytics: Collaboration, Rivalry and Fusion](https://mlsanity.com/paper/2006.08159/) (2020) - [Guidance and Evaluation: Semantic-Aware Image Inpainting for Mixed Scenes](https://mlsanity.com/paper/2003.06877/) (2020) - [Learning to Incorporate Structure Knowledge for Image Inpainting](https://mlsanity.com/paper/2002.04170/) (2020) - [Image Inpainting Guided by Coherence Priors of Semantics and Textures](https://mlsanity.com/paper/2012.08054/) (2020) - [Image Inpainting with External-internal Learning and Monochromic Bottleneck](https://mlsanity.com/paper/2104.09068/) (2021) ## Built on by (most cited first, 13 of 13) - [Towards Coherent Image Inpainting Using Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2304.03322/) (2023) - [Detecting and Grounding Multi-Modal Media Manipulation and Beyond](https://mlsanity.com/paper/2309.14203/) (2023) - [ERA-Solver: Error-Robust Adams Solver for Fast Sampling of Diffusion Probabilistic Models](https://mlsanity.com/paper/2301.12935/) (2023) - [Fine-grained Cross-modal Fusion based Refinement for Text-to-Image Synthesis](https://mlsanity.com/paper/2302.08706/) (2023) - [MDE-Edit: Masked Dual-Editing for Multi-Object Image Editing via Diffusion Models](https://mlsanity.com/paper/2505.05101/) (2025) - [Structure-Guided Image Completion with Image-level and Object-level Semantic Discriminators](https://mlsanity.com/paper/2212.06310/) (2022) - [From Missing Pieces to Masterpieces: Image Completion with Context-Adaptive Diffusion](https://mlsanity.com/paper/2504.14294/) (2025) - [DORS: Dynamic Attention Routing for Diffusion-based Object Removal in Dense Scenes](https://mlsanity.com/paper/2607.16656/) (2026) - [Domain Adaptive Person Search via GAN-based Scene Synthesis for Cross-scene Videos](https://mlsanity.com/paper/2308.04322/) (2023) - [Coarse-to-Fine Lightweight Meta-Embedding for ID-Based Recommendation](https://mlsanity.com/paper/2501.11870/) (2025) - [PIDiff: Image Customization for Personalized Identities with Diffusion Models](https://mlsanity.com/paper/2505.05081/) (2025) - [STEDiff: Strengthening Text Embedding for Text-to-Image Alignment in Diffusion Model](https://mlsanity.com/paper/2606.10653/) (2026) - [Rare Concept Generation via Counterfactual Inference in Diffusion Models](https://mlsanity.com/paper/2607.14765/) (2026) ## Abstract (arXiv) Image inpainting has achieved remarkable progress and inspired abundant methods, where the critical bottleneck is identified as how to fulfill the high-frequency structure and low-frequency texture information on the masked regions with semantics. To this end, deep models exhibit powerful superiority to capture them, yet constrained on the local spatial regions. In this paper, we delve globally into texture and structure information to well capture the semantics for image inpainting. As opposed to the existing arts trapped on the independent local patches, the texture information of each patch is reconstructed from all other patches across the whole image, to match the coarsely filled information, specially the structure information over the masked regions. Unlike the current decoder-only transformer within the pixel level for image inpainting, our model adopts the transformer pipeline paired with both encoder and decoder. On one hand, the encoder captures the texture semantic correlations of all patches across image via self-attention module. On the other hand, an adaptive patch vocabulary is dynamically established in the decoder for the filled patches over the masked regions. Building on this, a structure-texture matching attention module anchored on the known regions comes up to marry the best of these two worlds for progressive inpainting via a probabilistic diffusion process. Our model is orthogonal to the fashionable arts, such as Convolutional Neural Networks (CNNs), Attention and Transformer model, from the perspective of texture and structure information for image inpainting. The extensive experiments over the benchmarks validate its superiority. Our code is available at https://github.com/htyjers/DGTS-Inpainting.