# DAN: a Segmentation-free Document Attention Network for Handwritten Document Recognition

arXiv: https://arxiv.org/abs/2203.12273 · HTML: https://mlsanity.com/paper/2203.12273/

## Key facts

- Cited by 62 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 56 references that are in the map.
- First posted on arXiv 23 Mar 2022.

## Builds on (most cited first, 30 of 33)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [An End-to-End Trainable Neural Network for Image-based Sequence Recognition and Its Application to Scene Text Recognition](https://mlsanity.com/paper/1507.05717/) (2015)
- [LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understanding](https://mlsanity.com/paper/2012.14740/) (2020)
- [TrOCR: Transformer-based Optical Character Recognition with Pre-trained Models](https://mlsanity.com/paper/2109.10282/) (2021)
- [Going Full-TILT Boogie on Document Understanding with Text-Image-Layout Transformer](https://mlsanity.com/paper/2102.09550/) (2021)
- [Chargrid: Towards Understanding 2D Documents](https://mlsanity.com/paper/1809.08799/) (2018)
- [Learning to Extract Semantic Structure from Documents Using Multimodal Fully Convolutional Neural Network](https://mlsanity.com/paper/1706.02337/) (2017)
- [Vision Transformer for Fast and Efficient Scene Text Recognition](https://mlsanity.com/paper/2105.08582/) (2021)
- [Pay Attention to What You Read: Non-recurrent Handwritten Text-Line Recognition](https://mlsanity.com/paper/2005.13044/) (2020)
- [BERTgrid: Contextualized Embedding for 2D Document Representation and Understanding](https://mlsanity.com/paper/1909.04948/) (2019)
- [Evaluating Sequence-to-Sequence Models for Handwritten Text Recognition](https://mlsanity.com/paper/1903.07377/) (2019)
- [OrigamiNet: Weakly-Supervised, Segmentation-Free, One-Step, Full Page Text Recognition by learning to unfold](https://mlsanity.com/paper/2006.07491/) (2020)
- [Curriculum Dropout](https://mlsanity.com/paper/1703.06229/) (2017)
- [End-to-end Handwritten Paragraph Text Recognition Using a Vertical Attention Network](https://mlsanity.com/paper/2012.03868/) (2020)
- [Joint Line Segmentation and Transcription for End-to-End Handwritten Paragraph Recognition](https://mlsanity.com/paper/1604.08352/) (2016)
- [Scan, Attend and Read: End-to-End Handwritten Paragraph Recognition with MDLSTM Attention](https://mlsanity.com/paper/1604.03286/) (2016)
- [Handwritten Mathematical Expression Recognition with Bidirectionally Trained Transformer](https://mlsanity.com/paper/2105.02412/) (2021)
- [dhSegment: A generic deep-learning approach for document segmentation](https://mlsanity.com/paper/1804.10371/) (2018)
- [A Two-Stage Method for Text Line Detection in Historical Documents](https://mlsanity.com/paper/1802.03345/) (2018)
- [Full Page Handwriting Recognition via Image to Sequence Extraction](https://mlsanity.com/paper/2103.06450/) (2021)
- [Recurrence-free unconstrained handwritten text recognition using gated fully convolutional network](https://mlsanity.com/paper/2012.04961/) (2020)
- [SPAN: a Simple Predict & Align Network for Handwritten Paragraph Recognition](https://mlsanity.com/paper/2102.08742/) (2021)
- [DocSynth: A Layout Guided Approach for Controllable Document Image Synthesis](https://mlsanity.com/paper/2107.02638/) (2021)
- [Transformer-Based Approach for Joint Handwriting and Named Entity Recognition in Historical documents](https://mlsanity.com/paper/2112.04189/) (2021)
- [A Neural Model for Text Localization, Transcription and Named Entity Recognition in Full Pages](https://mlsanity.com/paper/1912.10016/) (2019)
- [Multiple Document Datasets Pre-training Improves Text Line Detection With Deep Neural Networks](https://mlsanity.com/paper/2012.14163/) (2020)

## Built on by (most cited first, 30 of 62)

- [Exploring OCR Capabilities of GPT-4V(ision) : A Quantitative and In-depth Evaluation](https://mlsanity.com/paper/2310.16809/) (2023)
- [Hi-SAM: Marrying Segment Anything Model for Hierarchical Text Segmentation](https://mlsanity.com/paper/2401.17904/) (2024)
- [HTR-VT: Handwritten Text Recognition with Vision Transformer](https://mlsanity.com/paper/2409.08573/) (2024)
- [Sheet Music Transformer: End-To-End Optical Music Recognition Beyond Monophonic Transcription](https://mlsanity.com/paper/2402.07596/) (2024)
- [RoDLA: Benchmarking the Robustness of Document Layout Analysis Models](https://mlsanity.com/paper/2403.14442/) (2024)
- [End-to-End Full-Page Optical Music Recognition for Pianoform Sheet Music](https://mlsanity.com/paper/2405.12105/) (2024)
- [Handwritten Text Recognition: A Survey](https://mlsanity.com/paper/2502.08417/) (2025)
- [Multimodal LLMs for OCR, OCR Post-Correction, and Named Entity Recognition in Historical Documents](https://mlsanity.com/paper/2504.00414/) (2025)
- [Benchmarking Large Language Models for Handwritten Text Recognition](https://mlsanity.com/paper/2503.15195/) (2025)
- [On the Generalization of Handwritten Text Recognition Models](https://mlsanity.com/paper/2411.17332/) (2024)
- [Faster DAN: Multi-target Queries with Document Positional Encoding for End-to-end Handwritten Document Recognition](https://mlsanity.com/paper/2301.10593/) (2023)
- [Key-value information extraction from full handwritten pages](https://mlsanity.com/paper/2304.13530/) (2023)
- [DANIEL: A fast Document Attention Network for Information Extraction and Labelling of handwritten documents](https://mlsanity.com/paper/2407.09103/) (2024)
- [End-to-End Page-Level Assessment of Handwritten Text Recognition](https://mlsanity.com/paper/2301.05935/) (2023)
- [Multi-Page Document Visual Question Answering using Self-Attention Scoring Mechanism](https://mlsanity.com/paper/2404.19024/) (2024)
- [MSdocTr-Lite: A Lite Transformer for Full Page Multi-script Handwriting Recognition](https://mlsanity.com/paper/2303.13931/) (2023)
- [SelfDocSeg: A Self-Supervised vision-based Approach towards Document Segmentation](https://mlsanity.com/paper/2305.00795/) (2023)
- [Improving Automatic Text Recognition with Language Models in the PyLaia Open-Source Library](https://mlsanity.com/paper/2404.18722/) (2024)
- [Large Scale Genealogical Information Extraction From Handwritten Quebec Parish Records](https://mlsanity.com/paper/2304.14044/) (2023)
- [The Socface Project: Large-Scale Collection, Processing, and Analysis of a Century of French Censuses](https://mlsanity.com/paper/2404.18706/) (2024)
- [End-to-end information extraction in handwritten documents: Understanding Paris marriage records from 1880 to 1940](https://mlsanity.com/paper/2404.19329/) (2024)
- [General Detection-based Text Line Recognition](https://mlsanity.com/paper/2409.17095/) (2024)
- [Self-Supervised Learning for Text Recognition: A Critical Survey](https://mlsanity.com/paper/2407.19889/) (2024)
- [DocTTT: Test-Time Training for Handwritten Document Recognition Using Meta-Auxiliary Learning](https://mlsanity.com/paper/2501.12898/) (2025)
- [UniHDSA: A Unified Relation Prediction Approach for Hierarchical Document Structure Analysis](https://mlsanity.com/paper/2503.15893/) (2025)
- [Meta-DAN: towards an efficient prediction strategy for page-level handwritten text recognition](https://mlsanity.com/paper/2504.03349/) (2025)
- [CM1 -- A Dataset for Evaluating Few-Shot Information Extraction with Large Vision Language Models](https://mlsanity.com/paper/2505.04214/) (2025)
- [Normalized vs Diplomatic Annotation: A Case Study of Automatic Information Extraction from Handwritten Uruguayan Birth Certificates](https://mlsanity.com/paper/2507.08636/) (2025)
- [Improving OCR using internal document redundancy](https://mlsanity.com/paper/2508.14557/) (2025)
- [GraphKD: Exploring Knowledge Distillation Towards Document Object Detection with Structured Graph Creation](https://mlsanity.com/paper/2402.11401/) (2024)

## Abstract (arXiv)

Unconstrained handwritten text recognition is a challenging computer vision task. It is traditionally handled by a two-step approach, combining line segmentation followed by text line recognition. For the first time, we propose an end-to-end segmentation-free architecture for the task of handwritten document recognition: the Document Attention Network. In addition to text recognition, the model is trained to label text parts using begin and end tags in an XML-like fashion. This model is made up of an FCN encoder for feature extraction and a stack of transformer decoder layers for a recurrent token-by-token prediction process. It takes whole text documents as input and sequentially outputs characters, as well as logical layout tokens. Contrary to the existing segmentation-based approaches, the model is trained without using any segmentation label. We achieve competitive results on the READ 2016 dataset at page level, as well as double-page level with a CER of 3.43% and 3.70%, respectively. We also provide results for the RIMES 2009 dataset at page level, reaching 4.54% of CER. We provide all source code and pre-trained model weights at https://github.com/FactoDeepLearning/DAN.