# Full Page Handwriting Recognition via Image to Sequence Extraction

arXiv: https://arxiv.org/abs/2103.06450 · HTML: https://mlsanity.com/paper/2103.06450/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 33 references that are in the map.
- First posted on arXiv 11 Mar 2021.

## Builds on (most cited first, 19 of 19)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [Pointer Sentinel Mixture Models](https://mlsanity.com/paper/1609.07843/) (2016)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context](https://mlsanity.com/paper/1901.02860/) (2019)
- [Show, Attend and Tell: Neural Image Caption Generation with Visual Attention](https://mlsanity.com/paper/1502.03044/) (2015)
- [Decoupled Attention Network for Text Recognition](https://mlsanity.com/paper/1912.10205/) (2019)
- [Tensor2Tensor for Neural Machine Translation](https://mlsanity.com/paper/1803.07416/) (2018)
- [Image-to-Markup Generation with Coarse-to-Fine Attention](https://mlsanity.com/paper/1609.04938/) (2016)
- [Pay Attention to What You Read: Non-recurrent Handwritten Text-Line Recognition](https://mlsanity.com/paper/2005.13044/) (2020)
- [Joint Line Segmentation and Transcription for End-to-End Handwritten Paragraph Recognition](https://mlsanity.com/paper/1604.08352/) (2016)
- [Scan, Attend and Read: End-to-End Handwritten Paragraph Recognition with MDLSTM Attention](https://mlsanity.com/paper/1604.03286/) (2016)
- [Dropout improves Recurrent Neural Networks for Handwriting Recognition](https://mlsanity.com/paper/1312.4569/) (2013)
- [Teaching Machines to Code: Neural Markup Generation with Visual Attention](https://mlsanity.com/paper/1802.05415/) (2018)

## Built on by (most cited first, 24 of 24)

- [DAN: a Segmentation-free Document Attention Network for Handwritten Document Recognition](https://mlsanity.com/paper/2203.12273/) (2022)
- [Sheet Music Transformer: End-To-End Optical Music Recognition Beyond Monophonic Transcription](https://mlsanity.com/paper/2402.07596/) (2024)
- [End-to-End Full-Page Optical Music Recognition for Pianoform Sheet Music](https://mlsanity.com/paper/2405.12105/) (2024)
- [Handwritten Text Recognition: A Survey](https://mlsanity.com/paper/2502.08417/) (2025)
- [Benchmarking Large Language Models for Handwritten Text Recognition](https://mlsanity.com/paper/2503.15195/) (2025)
- [Faster DAN: Multi-target Queries with Document Positional Encoding for End-to-end Handwritten Document Recognition](https://mlsanity.com/paper/2301.10593/) (2023)
- [DANIEL: A fast Document Attention Network for Information Extraction and Labelling of handwritten documents](https://mlsanity.com/paper/2407.09103/) (2024)
- [End-to-End Page-Level Assessment of Handwritten Text Recognition](https://mlsanity.com/paper/2301.05935/) (2023)
- [Text Image Inpainting via Global Structure-Guided Diffusion Models](https://mlsanity.com/paper/2401.14832/) (2024)
- [MSdocTr-Lite: A Lite Transformer for Full Page Multi-script Handwriting Recognition](https://mlsanity.com/paper/2303.13931/) (2023)
- [Auxiliary Cross-Modal Representation Learning with Triplet Loss Functions for Online Handwriting Recognition](https://mlsanity.com/paper/2202.07901/) (2022)
- [Self-Supervised Learning for Text Recognition: A Critical Survey](https://mlsanity.com/paper/2407.19889/) (2024)
- [DocTTT: Test-Time Training for Handwritten Document Recognition Using Meta-Auxiliary Learning](https://mlsanity.com/paper/2501.12898/) (2025)
- [Meta-DAN: towards an efficient prediction strategy for page-level handwritten text recognition](https://mlsanity.com/paper/2504.03349/) (2025)
- [Normalized vs Diplomatic Annotation: A Case Study of Automatic Information Extraction from Handwritten Uruguayan Birth Certificates](https://mlsanity.com/paper/2507.08636/) (2025)
- [From Robots to Books: An Introduction to Smart Applications of AI in Education (AIEd)](https://mlsanity.com/paper/2301.10026/) (2023)
- [Writer adaptation for offline text recognition: An exploration of neural network-based methods](https://mlsanity.com/paper/2307.15071/) (2023)
- [IDPL-PFOD2: A New Large-Scale Dataset for Printed Farsi Optical Character Recognition](https://mlsanity.com/paper/2312.01177/) (2023)
- [RVAFM: Re-parameterizing Vertical Attention Fusion Module for Handwritten Paragraph Text Recognition](https://mlsanity.com/paper/2503.03104/) (2025)
- [The OCR Quest for Generalization: Learning to recognize low-resource alphabets with model editing](https://mlsanity.com/paper/2506.06761/) (2025)
- [Transcoda: End-to-End Zero-Shot Optical Music Recognition via Data-Centric Synthetic Training](https://mlsanity.com/paper/2605.10835/) (2026)
- [Proceedings of the 4th International Workshop on Reading Music Systems](https://mlsanity.com/paper/2211.13285/) (2022)
- [Proceedings of the 5th International Workshop on Reading Music Systems](https://mlsanity.com/paper/2311.04091/) (2023)
- [Classification of Non-native Handwritten Characters Using Convolutional Neural Network](https://mlsanity.com/paper/2406.04511/) (2024)

## Abstract (arXiv)

We present a Neural Network based Handwritten Text Recognition (HTR) model architecture that can be trained to recognize full pages of handwritten or printed text without image segmentation. Being based on Image to Sequence architecture, it can extract text present in an image and then sequence it correctly without imposing any constraints regarding orientation, layout and size of text and non-text. Further, it can also be trained to generate auxiliary markup related to formatting, layout and content. We use character level vocabulary, thereby enabling language and terminology of any subject. The model achieves a new state-of-art in paragraph level recognition on the IAM dataset. When evaluated on scans of real world handwritten free form test answers - beset with curved and slanted lines, drawings, tables, math, chemistry and other symbols - it performs better than all commercially available HTR cloud APIs. It is deployed in production as part of a commercial web application.