# Scan, Attend and Read: End-to-End Handwritten Paragraph Recognition with MDLSTM Attention

arXiv: https://arxiv.org/abs/1604.03286 · HTML: https://mlsanity.com/paper/1604.03286/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Apr 2016.

## Built on by (most cited first, 30 of 45)

- [DAN: a Segmentation-free Document Attention Network for Handwritten Document Recognition](https://mlsanity.com/paper/2203.12273/) (2022)
- [Full Page Handwriting Recognition via Image to Sequence Extraction](https://mlsanity.com/paper/2103.06450/) (2021)
- [HTR-VT: Handwritten Text Recognition with Vision Transformer](https://mlsanity.com/paper/2409.08573/) (2024)
- [SPAN: a Simple Predict & Align Network for Handwritten Paragraph Recognition](https://mlsanity.com/paper/2102.08742/) (2021)
- [Transformer-Based Approach for Joint Handwriting and Named Entity Recognition in Historical documents](https://mlsanity.com/paper/2112.04189/) (2021)
- [The LAM Dataset: A Novel Benchmark for Line-Level Handwritten Text Recognition](https://mlsanity.com/paper/2208.07682/) (2022)
- [Boosting Modern and Historical Handwritten Text Recognition with Deformable Convolutions](https://mlsanity.com/paper/2208.08109/) (2022)
- [How to Choose Pretrained Handwriting Recognition Models for Single Writer Fine-Tuning](https://mlsanity.com/paper/2305.02593/) (2023)
- [AttentionHTR: Handwritten Text Recognition Based on Attention Encoder-Decoder Networks](https://mlsanity.com/paper/2201.09390/) (2022)
- [Handwritten Text Recognition: A Survey](https://mlsanity.com/paper/2502.08417/) (2025)
- [Benchmarking Large Language Models for Handwritten Text Recognition](https://mlsanity.com/paper/2503.15195/) (2025)
- [KOHTD: Kazakh Offline Handwritten Text Dataset](https://mlsanity.com/paper/2110.04075/) (2021)
- [Faster DAN: Multi-target Queries with Document Positional Encoding for End-to-end Handwritten Document Recognition](https://mlsanity.com/paper/2301.10593/) (2023)
- [DANIEL: A fast Document Attention Network for Information Extraction and Labelling of handwritten documents](https://mlsanity.com/paper/2407.09103/) (2024)
- [PageNet: Towards End-to-End Weakly Supervised Page-Level Handwritten Chinese Text Recognition](https://mlsanity.com/paper/2207.14807/) (2022)
- [MSdocTr-Lite: A Lite Transformer for Full Page Multi-script Handwriting Recognition](https://mlsanity.com/paper/2303.13931/) (2023)
- [Qalam : A Multimodal LLM for Arabic Optical Character and Handwriting Recognition](https://mlsanity.com/paper/2407.13559/) (2024)
- [Auxiliary Cross-Modal Representation Learning with Triplet Loss Functions for Online Handwriting Recognition](https://mlsanity.com/paper/2202.07901/) (2022)
- [Generating Synthetic Handwritten Historical Documents With OCR Constrained GANs](https://mlsanity.com/paper/2103.08236/) (2021)
- [Self-Training of Handwritten Word Recognition for Synthetic-to-Real Adaptation](https://mlsanity.com/paper/2206.03149/) (2022)
- [Lexicon and Attention based Handwritten Text Recognition System](https://mlsanity.com/paper/2209.04817/) (2022)
- [A Comprehensive Handwritten Paragraph Text Recognition System: LexiconNet](https://mlsanity.com/paper/2205.11018/) (2022)
- [Large Scale Genealogical Information Extraction From Handwritten Quebec Parish Records](https://mlsanity.com/paper/2304.14044/) (2023)
- [BN-HTRd: A Benchmark Dataset for Document Level Offline Bangla Handwritten Text Recognition (HTR) and Line Segmentation](https://mlsanity.com/paper/2206.08977/) (2022)
- [Meta-DAN: towards an efficient prediction strategy for page-level handwritten text recognition](https://mlsanity.com/paper/2504.03349/) (2025)
- [Normalized vs Diplomatic Annotation: A Case Study of Automatic Information Extraction from Handwritten Uruguayan Birth Certificates](https://mlsanity.com/paper/2507.08636/) (2025)
- [Applications of Machine Learning in Document Digitisation](https://mlsanity.com/paper/2102.03239/) (2021)
- [Recurrent neural network transducer for Japanese and Chinese offline handwritten text recognition](https://mlsanity.com/paper/2106.14459/) (2021)
- [Quo Vadis Handwritten Text Generation for Handwritten Text Recognition?](https://mlsanity.com/paper/2508.09936/) (2025)
- [Evaluation of a Region Proposal Architecture for Multi-task Document Layout Analysis](https://mlsanity.com/paper/2106.11797/) (2021)

## Abstract (arXiv)

We present an attention-based model for end-to-end handwriting recognition. Our system does not require any segmentation of the input paragraph. The model is inspired by the differentiable attention models presented recently for speech recognition, image captioning or translation. The main difference is the covert and overt attention, implemented as a multi-dimensional LSTM network. Our principal contribution towards handwriting recognition lies in the automatic transcription without a prior segmentation into lines, which was crucial in previous approaches. To the best of our knowledge this is the first successful attempt of end-to-end multi-line handwriting recognition. We carried out experiments on the well-known IAM Database. The results are encouraging and bring hope to perform full paragraph transcription in the near future.