# Deep Unrestricted Document Image Rectification

arXiv: https://arxiv.org/abs/2304.08796 · HTML: https://mlsanity.com/paper/2304.08796/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 77 references that are in the map.
- First posted on arXiv 18 Apr 2023.

## Builds on (most cited first, 28 of 28)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [DocVQA: A Dataset for VQA on Document Images](https://mlsanity.com/paper/2007.00398/) (2020)
- [FlowNet: Learning Optical Flow with Convolutional Networks](https://mlsanity.com/paper/1504.06852/) (2015)
- [U2-Net: Going Deeper with Nested U-Structure for Salient Object Detection](https://mlsanity.com/paper/2005.09007/) (2020)
- [OCR-free Document Understanding Transformer](https://mlsanity.com/paper/2111.15664/) (2021)
- [Attention Augmented Convolutional Networks](https://mlsanity.com/paper/1904.09925/) (2019)
- [Real-time Scene Text Detection with Differentiable Binarization](https://mlsanity.com/paper/1911.08947/) (2019)
- [DocTr: Document Image Transformer for Geometric Unwarping and Illumination Correction](https://mlsanity.com/paper/2110.12942/) (2021)
- [Document Rectification and Illumination Correction using a Patch-based CNN](https://mlsanity.com/paper/1909.09470/) (2019)
- [Multiview Rectification of Folded Documents](https://mlsanity.com/paper/1606.00166/) (2016)
- [Geometric Representation Learning for Document Image Rectification](https://mlsanity.com/paper/2210.08161/) (2022)
- [DocScanner: Robust Document Image Rectification with Progressive Learning](https://mlsanity.com/paper/2110.14968/) (2021)
- [Revisiting Document Image Dewarping by Grid Regularization](https://mlsanity.com/paper/2203.16850/) (2022)
- [Multimodal Pre-training Based on Graph Attention Network for Document Understanding](https://mlsanity.com/paper/2203.13530/) (2022)
- [Fourier Document Restoration for Robust Document Dewarping and Recognition](https://mlsanity.com/paper/2203.09910/) (2022)
- [Dewarping Document Image By Displacement Flow Estimation with Fully Convolutional Network](https://mlsanity.com/paper/2104.06815/) (2021)
- [Document Dewarping with Control Points](https://mlsanity.com/paper/2203.10543/) (2022)
- [Can You Read Me Now? Content Aware Rectification using Angle Supervision](https://mlsanity.com/paper/2008.02231/) (2020)
- [Marior: Margin Removal and Iterative Content Rectification for Document Dewarping in the Wild](https://mlsanity.com/paper/2207.11515/) (2022)
- [Recognition of Handwritten Chinese Text by Segmentation: A Segment-annotation-free Approach](https://mlsanity.com/paper/2207.14801/) (2022)
- [Intrinsic Decomposition of Document Images In-the-Wild](https://mlsanity.com/paper/2011.14447/) (2020)
- [Geometric Rectification of Creased Document Images based on Isometric Mapping](https://mlsanity.com/paper/2212.08365/) (2022)

## Built on by (most cited first, 16 of 16)

- [Visual Text Processing: A Comprehensive Review and Unified Evaluation](https://mlsanity.com/paper/2504.21682/) (2025)
- [SimFIR: A Simple Framework for Fisheye Image Rectification with Self-supervised Representation Learning](https://mlsanity.com/paper/2308.09040/) (2023)
- [RoDLA: Benchmarking the Robustness of Document Layout Analysis Models](https://mlsanity.com/paper/2403.14442/) (2024)
- [High-Fidelity Document Stain Removal via A Large-Scale Real-World Dataset and A Memory-Augmented Transformer](https://mlsanity.com/paper/2410.22922/) (2024)
- [Visual Text Meets Low-level Vision: A Comprehensive Survey on Visual Text Processing](https://mlsanity.com/paper/2402.03082/) (2024)
- [Text-Aware Image Restoration with Diffusion Models](https://mlsanity.com/paper/2506.09993/) (2025)
- [Unified Diffusion Transformer for High-fidelity Text-Aware Image Restoration](https://mlsanity.com/paper/2512.08922/) (2025)
- [Efficient Document Image Dewarping via Hybrid Deep Learning and Cubic Polynomial Geometry Restoration](https://mlsanity.com/paper/2501.03145/) (2025)
- [CHAOS: Chart Analysis with Outlier Samples](https://mlsanity.com/paper/2505.17235/) (2025)
- [Unfolder: Fast localization and image rectification of a document with a crease from folding in half](https://mlsanity.com/paper/2312.00467/) (2023)
- [D2Dewarp: Dual Dimensions Geometric Representation Learning Based Document Image Dewarping](https://mlsanity.com/paper/2507.08492/) (2025)
- [DocIQ: A Benchmark Dataset and Feature Fusion Network for Document Image Quality Assessment](https://mlsanity.com/paper/2509.17012/) (2025)
- [Cascaded Robust Rectification for Arbitrary Document Images](https://mlsanity.com/paper/2511.23150/) (2025)
- [SyntheticDoc: A Large Synthetic Dataset for Document Unwarping and Illumination Correction](https://mlsanity.com/paper/2609.15503/) (2026)
- [ENet-GP: Unified Document Image Restoration](https://mlsanity.com/paper/2609.33758/) (2026)
- [Rethinking Supervision in Document Unwarping: A Self-consistent Flow-free Approach](https://mlsanity.com/paper/2610.15445/) (2026)

## Abstract (arXiv)

In recent years, tremendous efforts have been made on document image rectification, but existing advanced algorithms are limited to processing restricted document images, i.e., the input images must incorporate a complete document. Once the captured image merely involves a local text region, its rectification quality is degraded and unsatisfactory. Our previously proposed DocTr, a transformer-assisted network for document image rectification, also suffers from this limitation. In this work, we present DocTr++, a novel unified framework for document image rectification, without any restrictions on the input distorted images. Our major technical improvements can be concluded in three aspects. Firstly, we upgrade the original architecture by adopting a hierarchical encoder-decoder structure for multi-scale representation extraction and parsing. Secondly, we reformulate the pixel-wise mapping relationship between the unrestricted distorted document images and the distortion-free counterparts. The obtained data is used to train our DocTr++ for unrestricted document image rectification. Thirdly, we contribute a real-world test set and metrics applicable for evaluating the rectification quality. To our best knowledge, this is the first learning-based method for the rectification of unrestricted document images. Extensive experiments are conducted, and the results demonstrate the effectiveness and superiority of our method. We hope our DocTr++ will serve as a strong baseline for generic document image rectification, prompting the further advancement and application of learning-based algorithms. The source code and the proposed dataset are publicly available at https://github.com/fh2019ustc/DocTr-Plus.