# DocDiff: Document Enhancement via Residual Diffusion Models

arXiv: https://arxiv.org/abs/2305.03892 · HTML: https://mlsanity.com/paper/2305.03892/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- Placed in Inverse problems and image restoration, one of 14 directions in Diffusion models.
- Builds on 24 of its 60 references that are in the map.
- First posted on arXiv 6 May 2023.

## Placed in

- Diffusion models / [Inverse problems and image restoration](https://mlsanity.com/diffusion/inverse-problems-and-image-restoration/)

## Builds on (most cited first, 24 of 24)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Image Super-Resolution via Iterative Refinement](https://mlsanity.com/paper/2104.07636/) (2021)
- [Palette: Image-to-Image Diffusion Models](https://mlsanity.com/paper/2111.05826/) (2021)
- [MUSIQ: Multi-scale Image Quality Transformer](https://mlsanity.com/paper/2108.05997/) (2021)
- [Multi-Stage Progressive Image Restoration](https://mlsanity.com/paper/2102.02808/) (2021)
- [Image Quality Assessment: Unifying Structure and Texture Similarity](https://mlsanity.com/paper/2004.07728/) (2020)
- [The Perception-Distortion Tradeoff](https://mlsanity.com/paper/1711.06077/) (2017)
- [Zero-Shot Image Restoration Using Denoising Diffusion Null-Space Model](https://mlsanity.com/paper/2212.00490/) (2022)
- [MANIQA: Multi-dimension Attention Network for No-Reference Image Quality Assessment](https://mlsanity.com/paper/2204.08958/) (2022)
- [SRDiff: Single Image Super-Resolution with Diffusion Probabilistic Models](https://mlsanity.com/paper/2104.14951/) (2021)
- [DeblurGAN-v2: Deblurring (Orders-of-Magnitude) Faster and Better](https://mlsanity.com/paper/1908.03826/) (2019)
- [Deblurring via Stochastic Refinement](https://mlsanity.com/paper/2112.02475/) (2021)
- [HINet: Half Instance Normalization Network for Image Restoration](https://mlsanity.com/paper/2105.06086/) (2021)
- [MedSegDiff: Medical Image Segmentation with Diffusion Probabilistic Model](https://mlsanity.com/paper/2211.00611/) (2022)
- [Frequency Separation for Real-World Super-Resolution](https://mlsanity.com/paper/1911.07850/) (2019)
- [Fourier Space Losses for Efficient Perceptual Image Super-Resolution](https://mlsanity.com/paper/2106.00783/) (2021)
- [ResDiff: Combining CNN and Diffusion Model for Image Super-Resolution](https://mlsanity.com/paper/2303.08714/) (2023)
- [DE-GAN: A Conditional Generative Adversarial Network for Document Enhancement](https://mlsanity.com/paper/2010.08764/) (2020)
- [NTIRE 2022 Challenge on Perceptual Image Quality Assessment](https://mlsanity.com/paper/2206.11695/) (2022)
- [DocEnTr: An End-to-End Document Image Enhancement Transformer](https://mlsanity.com/paper/2201.10252/) (2022)
- [CDPMSR: Conditional Diffusion Probabilistic Models for Single Image Super-Resolution](https://mlsanity.com/paper/2302.12831/) (2023)

## Built on by (most cited first, 30 of 37)

- [Wavelet-based Fourier Information Interaction with Frequency Diffusion Adjustment for Underwater Image Restoration](https://mlsanity.com/paper/2311.16845/) (2023)
- [DocRes: A Generalist Model Toward Unifying Document Image Restoration Tasks](https://mlsanity.com/paper/2405.04408/) (2024)
- [Learning A Physical-aware Diffusion Model Based on Transformer for Underwater Image Enhancement](https://mlsanity.com/paper/2403.01497/) (2024)
- [Visual Text Processing: A Comprehensive Review and Unified Evaluation](https://mlsanity.com/paper/2504.21682/) (2025)
- [High-Fidelity Document Stain Removal via A Large-Scale Real-World Dataset and A Memory-Augmented Transformer](https://mlsanity.com/paper/2410.22922/) (2024)
- [TextDiff: Mask-Guided Residual Diffusion Models for Scene Text Image Super-Resolution](https://mlsanity.com/paper/2308.06743/) (2023)
- [NAF-DPM: A Nonlinear Activation-Free Diffusion Probabilistic Model for Document Enhancement](https://mlsanity.com/paper/2404.05669/) (2024)
- [Predicting the Original Appearance of Damaged Historical Documents](https://mlsanity.com/paper/2412.11634/) (2024)
- [Visual Text Meets Low-level Vision: A Comprehensive Survey on Visual Text Processing](https://mlsanity.com/paper/2402.03082/) (2024)
- [DocEdit-v2: Document Structure Editing Via Multimodal LLM Grounding](https://mlsanity.com/paper/2410.16472/) (2024)
- [Uni-DocDiff: A Unified Document Restoration Model Based on Diffusion](https://mlsanity.com/paper/2508.04055/) (2025)
- [PEAN: A Diffusion-Based Prior-Enhanced Attention Network for Scene Text Image Super-Resolution](https://mlsanity.com/paper/2311.17955/) (2023)
- [DocBinFormer: A Two-Level Transformer Network for Effective Document Image Binarization](https://mlsanity.com/paper/2312.03568/) (2023)
- [A Comprehensive Survey on Diffusion Models and Their Applications](https://mlsanity.com/paper/2408.10207/) (2024)
- [DocStormer: Revitalizing Multi-Degraded Colored Document Images to Pristine PDF](https://mlsanity.com/paper/2310.17910/) (2023)
- [Binarizing Documents by Leveraging both Space and Frequency](https://mlsanity.com/paper/2404.17243/) (2024)
- [Adaptive Semantic-Enhanced Denoising Diffusion Probabilistic Model for Remote Sensing Image Super-Resolution](https://mlsanity.com/paper/2403.11078/) (2024)
- [Latent Diffusion for Guided Document Table Generation](https://mlsanity.com/paper/2408.09800/) (2024)
- [TextDoctor: Unified Document Image Inpainting via Patch Pyramid Diffusion Models](https://mlsanity.com/paper/2503.04021/) (2025)
- [DKDS: A Benchmark Dataset of Degraded Kuzushiji Documents with Seals for Detection and Binarization](https://mlsanity.com/paper/2511.09117/) (2025)
- [MFE-GAN: Efficient GAN-based Framework for Document Image Enhancement and Binarization with Multi-scale Feature Extraction](https://mlsanity.com/paper/2512.14114/) (2025)
- [R3D: Regional-guided Residual Radar Diffusion](https://mlsanity.com/paper/2601.06465/) (2026)
- [EpiAgent: An Agent-Centric System for Ancient Inscription Restoration](https://mlsanity.com/paper/2604.09367/) (2026)
- [Seal2Real: Prompt Prior Learning on Diffusion Model for Unsupervised Document Seal Data Generation and Realisation](https://mlsanity.com/paper/2310.00546/) (2023)
- [DeepMine-Mamba: Mitigating Information Dilution in Mamba-Based State Space Models for Document Image Binarization](https://mlsanity.com/paper/2606.08781/) (2026)
- [Technique Report of CVPR 2024 PBDL Challenges](https://mlsanity.com/paper/2406.10744/) (2024)
- [Single Document Image Highlight Removal via A Large-Scale Real-World Dataset and A Location-Aware Network](https://mlsanity.com/paper/2504.14238/) (2025)
- [GL-PGENet: A Parameterized Generation Framework for Robust Document Image Enhancement](https://mlsanity.com/paper/2505.22021/) (2025)
- [DocShaDiffusion: Diffusion Model in Latent Space for Document Image Shadow Removal](https://mlsanity.com/paper/2507.01422/) (2025)
- [D2Dewarp: Dual Dimensions Geometric Representation Learning Based Document Image Dewarping](https://mlsanity.com/paper/2507.08492/) (2025)

## Abstract (arXiv)

Removing degradation from document images not only improves their visual quality and readability, but also enhances the performance of numerous automated document analysis and recognition tasks. However, existing regression-based methods optimized for pixel-level distortion reduction tend to suffer from significant loss of high-frequency information, leading to distorted and blurred text edges. To compensate for this major deficiency, we propose DocDiff, the first diffusion-based framework specifically designed for diverse challenging document enhancement problems, including document deblurring, denoising, and removal of watermarks and seals. DocDiff consists of two modules: the Coarse Predictor (CP), which is responsible for recovering the primary low-frequency content, and the High-Frequency Residual Refinement (HRR) module, which adopts the diffusion models to predict the residual (high-frequency information, including text edges), between the ground-truth and the CP-predicted image. DocDiff is a compact and computationally efficient model that benefits from a well-designed network architecture, an optimized training loss objective, and a deterministic sampling process with short time steps. Extensive experiments demonstrate that DocDiff achieves state-of-the-art (SOTA) performance on multiple benchmark datasets, and can significantly enhance the readability and recognizability of degraded document images. Furthermore, our proposed HRR module in pre-trained DocDiff is plug-and-play and ready-to-use, with only 4.17M parameters. It greatly sharpens the text edges generated by SOTA deblurring methods without additional joint training. Available codes: https://github.com/Royalvice/DocDiff