# TextDoctor: Unified Document Image Inpainting via Patch Pyramid Diffusion Models

arXiv: https://arxiv.org/abs/2503.04021 · HTML: https://mlsanity.com/paper/2503.04021/

## Key facts

- Cited by 2 papers in the map (data through 7 Oct 2026).
- Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models.
- Builds on 33 of its 54 references that are in the map.
- First posted on arXiv 6 Mar 2025.

## Placed in

- Diffusion models / [3D, point cloud and scene generation](https://mlsanity.com/diffusion/3d-point-cloud-and-scene-generation/)

## Builds on (most cited first, 30 of 33)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [MultiDiffusion: Fusing Diffusion Paths for Controlled Image Generation](https://mlsanity.com/paper/2302.08113/) (2023)
- [An End-to-End Trainable Neural Network for Image-based Sequence Recognition and Its Application to Scene Text Recognition](https://mlsanity.com/paper/1507.05717/) (2015)
- [A Neural Algorithm of Artistic Style](https://mlsanity.com/paper/1508.06576/) (2015)
- [FUNSD: A Dataset for Form Understanding in Noisy Scanned Documents](https://mlsanity.com/paper/1905.13538/) (2019)
- [VM-UNet: Vision Mamba UNet for Medical Image Segmentation](https://mlsanity.com/paper/2402.02491/) (2024)
- [Large Scale Image Completion via Co-Modulated Generative Adversarial Networks](https://mlsanity.com/paper/2103.10428/) (2021)
- [AnyText: Multilingual Visual Text Generation And Editing](https://mlsanity.com/paper/2311.03054/) (2023)
- [Contextual Residual Aggregation for Ultra High-Resolution Image Inpainting](https://mlsanity.com/paper/2005.09704/) (2020)
- [SyncDiffusion: Coherent Montage via Synchronized Joint Diffusions](https://mlsanity.com/paper/2306.05178/) (2023)
- [DemoFusion: Democratising High-Resolution Image Generation With No ](https://mlsanity.com/paper/2311.16973/) (2023)
- [Aggregated Contextual Transformations for High-Resolution Image Inpainting](https://mlsanity.com/paper/2104.01431/) (2021)
- [DE-GAN: A Conditional Generative Adversarial Network for Document Enhancement](https://mlsanity.com/paper/2010.08764/) (2020)
- [DocDiff: Document Enhancement via Residual Diffusion Models](https://mlsanity.com/paper/2305.03892/) (2023)
- [High-Resolution Document Shadow Removal via A Large-Scale Real-World Dataset and A Frequency-Aware Shadow Erasing Net](https://mlsanity.com/paper/2308.14221/) (2023)
- [CoordFill: Efficient High-Resolution Image Inpainting via Parameterized Coordinate Querying](https://mlsanity.com/paper/2303.08524/) (2023)
- [DocEnTr: An End-to-End Document Image Enhancement Transformer](https://mlsanity.com/paper/2201.10252/) (2022)
- [DocRes: A Generalist Model Toward Unifying Document Image Restoration Tasks](https://mlsanity.com/paper/2405.04408/) (2024)
- [Geometric Representation Learning for Document Image Rectification](https://mlsanity.com/paper/2210.08161/) (2022)
- [Roboflow 100: A Rich, Multi-Domain Object Detection Benchmark](https://mlsanity.com/paper/2211.13523/) (2022)
- [Restoring Vision in Hazy Weather with Hierarchical Contrastive Learning](https://mlsanity.com/paper/2212.11473/) (2022)
- [PromptRR: Diffusion Models as Prompt Generators for Single Image Reflection Removal](https://mlsanity.com/paper/2402.02374/) (2024)
- [TextDiff: Mask-Guided Residual Diffusion Models for Scene Text Image Super-Resolution](https://mlsanity.com/paper/2308.06743/) (2023)
- [GDB: Gated convolutions-based Document Binarization](https://mlsanity.com/paper/2302.02073/) (2023)
- [NAF-DPM: A Nonlinear Activation-Free Diffusion Probabilistic Model for Document Enhancement](https://mlsanity.com/paper/2404.05669/) (2024)
- [Text Image Inpainting via Global Structure-Guided Diffusion Models](https://mlsanity.com/paper/2401.14832/) (2024)
- [FACEMUG: A Multimodal Generative and Fusion Framework for Local Facial Editing](https://mlsanity.com/paper/2412.19009/) (2024)
- [A novel segmentation dataset for signatures on bank checks](https://mlsanity.com/paper/2104.12203/) (2021)

## Built on by (most cited first, 2 of 2)

- [DocPure: Prompt-Free Unified Document Restoration via Degradation-Aware Structure-Guided Wavelet Modulation](https://mlsanity.com/paper/2608.09536/) (2026)
- [Beyond Pixel Reconstruction: Retrieval-Guided Glyph-Aware Restoration for Low-Resource Manchu Historical Documents](https://mlsanity.com/paper/2610.00315/) (2026)

## Abstract (arXiv)

Digital versions of real-world text documents often suffer from issues like environmental corrosion of the original document, low-quality scanning, or human interference. Existing document restoration and inpainting methods typically struggle with generalizing to unseen document styles and handling high-resolution images. To address these challenges, we introduce TextDoctor, a novel unified document image inpainting method. Inspired by human reading behavior, TextDoctor restores fundamental text elements from patches and then applies diffusion models to entire document images instead of training models on specific document types. To handle varying text sizes and avoid out-of-memory issues, common in high-resolution documents, we propose using structure pyramid prediction and patch pyramid diffusion models. These techniques leverage multiscale inputs and pyramid patches to enhance the quality of inpainting both globally and locally. Extensive qualitative and quantitative experiments on seven public datasets validated that TextDoctor outperforms state-of-the-art methods in restoring various types of high-resolution document images.