# Text Image Inpainting via Global Structure-Guided Diffusion Models arXiv: https://arxiv.org/abs/2401.14832 ยท HTML: https://mlsanity.com/paper/2401.14832/ ## Key facts - Cited by 8 papers in the map (data through 14 Oct 2026). - Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. - Builds on 36 of its 55 references that are in the map. - First posted on arXiv 26 Jan 2024. ## Placed in - Diffusion models / [3D, point cloud and scene generation](https://mlsanity.com/diffusion/3d-point-cloud-and-scene-generation/) ## Builds on (most cited first, 30 of 36) - [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018) - [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014) - [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020) - [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015) - [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021) - [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020) - [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023) - [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022) - [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019) - [High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs](https://mlsanity.com/paper/1711.11585/) (2017) - [An End-to-End Trainable Neural Network for Image-based Sequence Recognition and Its Application to Scene Text Recognition](https://mlsanity.com/paper/1507.05717/) (2015) - [Image Inpainting for Irregular Holes Using Partial Convolutions](https://mlsanity.com/paper/1804.07723/) (2018) - [A Neural Algorithm of Artistic Style](https://mlsanity.com/paper/1508.06576/) (2015) - [Synthetic Data for Text Localisation in Natural Images](https://mlsanity.com/paper/1604.06646/) (2016) - [Dilated Residual Networks](https://mlsanity.com/paper/1705.09914/) (2017) - [DiffIR: Efficient Diffusion Model for Image Restoration](https://mlsanity.com/paper/2303.09472/) (2023) - [Inpaint Anything: Segment Anything Meets Image Inpainting](https://mlsanity.com/paper/2304.06790/) (2023) - [Exploiting Deep Generative Prior for Versatile Image Restoration and Manipulation](https://mlsanity.com/paper/2003.13659/) (2020) - [Synthetic Data and Artificial Neural Networks for Natural Scene Text Recognition](https://mlsanity.com/paper/1406.2227/) (2014) - [Generative Diffusion Prior for Unified Image Restoration and Enhancement](https://mlsanity.com/paper/2304.01247/) (2023) - [Read Like Humans: Autonomous, Bidirectional and Iterative Language Modeling for Scene Text Recognition](https://mlsanity.com/paper/2103.06495/) (2021) - [High-Fidelity Pluralistic Image Completion with Transformers](https://mlsanity.com/paper/2103.14031/) (2021) - [Scene Text Detection and Recognition: The Deep Learning Era](https://mlsanity.com/paper/1811.04256/) (2018) - [Decoupled Attention Network for Text Recognition](https://mlsanity.com/paper/1912.10205/) (2019) - [Editing Text in the Wild](https://mlsanity.com/paper/1908.03047/) (2019) - [Scene Text Image Super-Resolution in the Wild](https://mlsanity.com/paper/2005.03341/) (2020) - [Towards Coherent Image Inpainting Using Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2304.03322/) (2023) - [Conditional Text Image Generation with Diffusion Models](https://mlsanity.com/paper/2306.10804/) (2023) - [Improving Diffusion Models for Scene Text Editing with Dual Encoders](https://mlsanity.com/paper/2304.05568/) (2023) - [ICL-D3IE: In-Context Learning with Diverse Demonstrations Updating for Document Information Extraction](https://mlsanity.com/paper/2303.05063/) (2023) ## Built on by (most cited first, 8 of 8) - [TextDoctor: Unified Document Image Inpainting via Patch Pyramid Diffusion Models](https://mlsanity.com/paper/2503.04021/) (2025) - [EpiAgent: An Agent-Centric System for Ancient Inscription Restoration](https://mlsanity.com/paper/2604.09367/) (2026) - [Structure-Guided Diffusion Models for High-Fidelity Portrait Shadow Removal](https://mlsanity.com/paper/2507.04692/) (2025) - [KAO: Kernel-Adaptive Optimization in Diffusion for Satellite Image](https://mlsanity.com/paper/2511.02462/) (2025) - [DocRevive: A Unified Pipeline for Document Text Restoration](https://mlsanity.com/paper/2604.10077/) (2026) - [Kernel Reboot: Breaking the Boundaries of Neural Tangent Kernels for Neural Fields](https://mlsanity.com/paper/2609.03117/) (2026) - [Think Before You Restore: Risk-Aware Manchu Manuscript Restoration with Stroke-Guided Attention](https://mlsanity.com/paper/2609.36243/) (2026) - [Beyond Pixel Reconstruction: Retrieval-Guided Glyph-Aware Restoration for Low-Resource Manchu Historical Documents](https://mlsanity.com/paper/2610.00315/) (2026) ## Abstract (arXiv) Real-world text can be damaged by corrosion issues caused by environmental or human factors, which hinder the preservation of the complete styles of texts, e.g., texture and structure. These corrosion issues, such as graffiti signs and incomplete signatures, bring difficulties in understanding the texts, thereby posing significant challenges to downstream applications, e.g., scene text recognition and signature identification. Notably, current inpainting techniques often fail to adequately address this problem and have difficulties restoring accurate text images along with reasonable and consistent styles. Formulating this as an open problem of text image inpainting, this paper aims to build a benchmark to facilitate its study. In doing so, we establish two specific text inpainting datasets which contain scene text images and handwritten text images, respectively. Each of them includes images revamped by real-life and synthetic datasets, featuring pairs of original images, corrupted images, and other assistant information. On top of the datasets, we further develop a novel neural framework, Global Structure-guided Diffusion Model (GSDM), as a potential solution. Leveraging the global structure of the text as a prior, the proposed GSDM develops an efficient diffusion model to recover clean texts. The efficacy of our approach is demonstrated by thorough empirical study, including a substantial boost in both recognition accuracy and image quality. These findings not only highlight the effectiveness of our method but also underscore its potential to enhance the broader field of text image understanding and processing. Code and datasets are available at: https://github.com/blackprotoss/GSDM.