# DocRes: A Generalist Model Toward Unifying Document Image Restoration Tasks

arXiv: https://arxiv.org/abs/2405.04408 · HTML: https://mlsanity.com/paper/2405.04408/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 74 references that are in the map.
- First posted on arXiv 7 May 2024.

## Builds on (most cited first, 30 of 34)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Restormer: Efficient Transformer for High-Resolution Image Restoration](https://mlsanity.com/paper/2111.09881/) (2021)
- [OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Framework](https://mlsanity.com/paper/2202.03052/) (2022)
- [Unified-IO: A Unified Model for Vision, Language, and Multi-Modal Tasks](https://mlsanity.com/paper/2206.08916/) (2022)
- [Images Speak in Images: A Generalist Painter for In-Context Visual Learning](https://mlsanity.com/paper/2212.02499/) (2022)
- [An Intriguing Failing of Convolutional Neural Networks and the CoordConv Solution](https://mlsanity.com/paper/1807.03247/) (2018)
- [Visual Prompting via Image Inpainting](https://mlsanity.com/paper/2209.00647/) (2022)
- [TransWeather: Transformer-based Restoration of Images Degraded by Adverse Weather Conditions](https://mlsanity.com/paper/2111.14813/) (2021)
- [A Unified Sequence Interface for Vision Tasks](https://mlsanity.com/paper/2206.07669/) (2022)
- [Unifying Vision, Text, and Layout for Universal Document Processing](https://mlsanity.com/paper/2212.02623/) (2022)
- [PromptIR: Prompting for All-in-One Blind Image Restoration](https://mlsanity.com/paper/2306.13090/) (2023)
- [ProRes: Exploring Degradation-aware Visual Prompt for Universal Image Restoration](https://mlsanity.com/paper/2306.13653/) (2023)
- [M6Doc: A Large-Scale Multi-Format, Multi-Type, Multi-Layout, Multi-Language, Multi-Annotation Category Dataset for Modern Document Layout Analysis](https://mlsanity.com/paper/2305.08719/) (2023)
- [DE-GAN: A Conditional Generative Adversarial Network for Document Enhancement](https://mlsanity.com/paper/2010.08764/) (2020)
- [DocTr: Document Image Transformer for Geometric Unwarping and Illumination Correction](https://mlsanity.com/paper/2110.12942/) (2021)
- [DocDiff: Document Enhancement via Residual Diffusion Models](https://mlsanity.com/paper/2305.03892/) (2023)
- [High-Resolution Document Shadow Removal via A Large-Scale Real-World Dataset and A Frequency-Aware Shadow Erasing Net](https://mlsanity.com/paper/2308.14221/) (2023)
- [Multiview Rectification of Folded Documents](https://mlsanity.com/paper/1606.00166/) (2016)
- [Unifying Image Processing as Visual Prompting Question Answering](https://mlsanity.com/paper/2310.10513/) (2023)
- [Geometric Representation Learning for Document Image Rectification](https://mlsanity.com/paper/2210.08161/) (2022)
- [Document Image Binarization with Fully Convolutional Neural Networks](https://mlsanity.com/paper/1708.03276/) (2017)
- [DocScanner: Robust Document Image Rectification with Progressive Learning](https://mlsanity.com/paper/2110.14968/) (2021)
- [Revisiting Document Image Dewarping by Grid Regularization](https://mlsanity.com/paper/2203.16850/) (2022)
- [All-in-one Multi-degradation Image Restoration Network via Hierarchical Degradation Representation](https://mlsanity.com/paper/2308.03021/) (2023)
- [Document Dewarping with Control Points](https://mlsanity.com/paper/2203.10543/) (2022)
- [Water-Filling: An Efficient Algorithm for Digitized Document Shadow Removal](https://mlsanity.com/paper/1904.09763/) (2019)
- [Text-DIAE: A Self-Supervised Degradation Invariant Autoencoders for Text Recognition and Document Enhancement](https://mlsanity.com/paper/2203.04814/) (2022)
- [Marior: Margin Removal and Iterative Content Rectification for Document Dewarping in the Wild](https://mlsanity.com/paper/2207.11515/) (2022)
- [UDoc-GAN: Unpaired Document Illumination Correction with Background Light Prior](https://mlsanity.com/paper/2210.08216/) (2022)
- [Image Restoration by Estimating Frequency Distribution of Local Patches](https://mlsanity.com/paper/1805.09097/) (2018)

## Built on by (most cited first, 27 of 27)

- [A Survey on All-in-One Image Restoration: Taxonomy, Evaluation and Future Trends](https://mlsanity.com/paper/2410.15067/) (2024)
- [OmniParser V2: Structured-Points-of-Thought for Unified Visual Text Parsing and Its Generality to Multimodal Large Language Models](https://mlsanity.com/paper/2502.16161/) (2025)
- [Visual Text Processing: A Comprehensive Review and Unified Evaluation](https://mlsanity.com/paper/2504.21682/) (2025)
- [High-Fidelity Document Stain Removal via A Large-Scale Real-World Dataset and A Memory-Augmented Transformer](https://mlsanity.com/paper/2410.22922/) (2024)
- [Real5-OmniDocBench: A Full-Scale Physical Reconstruction Benchmark for Robust Document Parsing in the Wild](https://mlsanity.com/paper/2603.04205/) (2026)
- [OCRGenBench: A Comprehensive Benchmark for Evaluating OCR Generative Capabilities](https://mlsanity.com/paper/2507.15085/) (2025)
- [Uni-DocDiff: A Unified Document Restoration Model Based on Diffusion](https://mlsanity.com/paper/2508.04055/) (2025)
- [DocSAM: Unified Document Image Segmentation via Query Decomposition and Heterogeneous Mixed Learning](https://mlsanity.com/paper/2504.04085/) (2025)
- [DvD: Unleashing a Generative Paradigm for Document Dewarping via Coordinates-based Diffusion Model](https://mlsanity.com/paper/2505.21975/) (2025)
- [PromptSR: Cascade Prompting for Lightweight Image Super-Resolution](https://mlsanity.com/paper/2507.04118/) (2025)
- [TextDoctor: Unified Document Image Inpainting via Patch Pyramid Diffusion Models](https://mlsanity.com/paper/2503.04021/) (2025)
- [EpiAgent: An Agent-Centric System for Ancient Inscription Restoration](https://mlsanity.com/paper/2604.09367/) (2026)
- [LayeredDoc: Domain Adaptive Document Restoration with a Layer Separation Approach](https://mlsanity.com/paper/2406.08610/) (2024)
- [Leveraging Contrast Information for Efficient Document Shadow Removal](https://mlsanity.com/paper/2504.00385/) (2025)
- [XY-Cut++: Advanced Layout Ordering via Hierarchical Mask Mechanism on a Novel Benchmark](https://mlsanity.com/paper/2504.10258/) (2025)
- [ForCenNet: Foreground-Centric Network for Document Image Rectification](https://mlsanity.com/paper/2507.19804/) (2025)
- [Document Image Rectification Bases on Self-Adaptive Multitask Fusion](https://mlsanity.com/paper/2505.06038/) (2025)
- [GL-PGENet: A Parameterized Generation Framework for Robust Document Image Enhancement](https://mlsanity.com/paper/2505.22021/) (2025)
- [D2Dewarp: Dual Dimensions Geometric Representation Learning Based Document Image Dewarping](https://mlsanity.com/paper/2507.08492/) (2025)
- [DocIQ: A Benchmark Dataset and Feature Fusion Network for Document Image Quality Assessment](https://mlsanity.com/paper/2509.17012/) (2025)
- [Cascaded Robust Rectification for Arbitrary Document Images](https://mlsanity.com/paper/2511.23150/) (2025)
- [DocRevive: A Unified Pipeline for Document Text Restoration](https://mlsanity.com/paper/2604.10077/) (2026)
- [DocPure: Prompt-Free Unified Document Restoration via Degradation-Aware Structure-Guided Wavelet Modulation](https://mlsanity.com/paper/2608.09536/) (2026)
- [DocIntent: Answerability-Guided Agentic Restoration for Real-World Document Visual Question Answering](https://mlsanity.com/paper/2608.29037/) (2026)
- [SyntheticDoc: A Large Synthetic Dataset for Document Unwarping and Illumination Correction](https://mlsanity.com/paper/2609.15503/) (2026)
- [ENet-GP: Unified Document Image Restoration](https://mlsanity.com/paper/2609.33758/) (2026)
- [Think Before You Restore: Risk-Aware Manchu Manuscript Restoration with Stroke-Guided Attention](https://mlsanity.com/paper/2609.36243/) (2026)

## Abstract (arXiv)

Document image restoration is a crucial aspect of Document AI systems, as the quality of document images significantly influences the overall performance. Prevailing methods address distinct restoration tasks independently, leading to intricate systems and the incapability to harness the potential synergies of multi-task learning. To overcome this challenge, we propose DocRes, a generalist model that unifies five document image restoration tasks including dewarping, deshadowing, appearance enhancement, deblurring, and binarization. To instruct DocRes to perform various restoration tasks, we propose a novel visual prompt approach called Dynamic Task-Specific Prompt (DTSPrompt). The DTSPrompt for different tasks comprises distinct prior features, which are additional characteristics extracted from the input image. Beyond its role as a cue for task-specific execution, DTSPrompt can also serve as supplementary information to enhance the model's performance. Moreover, DTSPrompt is more flexible than prior visual prompt approaches as it can be seamlessly applied and adapted to inputs with high and variable resolutions. Experimental results demonstrate that DocRes achieves competitive or superior performance compared to existing state-of-the-art task-specific models. This underscores the potential of DocRes across a broader spectrum of document image restoration tasks. The source code is publicly available at https://github.com/ZZZHANG-jx/DocRes