# End-to-end Handwritten Paragraph Text Recognition Using a Vertical Attention Network

arXiv: https://arxiv.org/abs/2012.03868 · HTML: https://mlsanity.com/paper/2012.03868/

## Key facts

- Cited by 50 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Dec 2020.

## Built on by (most cited first, 30 of 50)

- [End-to-end Document Recognition and Understanding with Dessurt](https://mlsanity.com/paper/2203.16618/) (2022)
- [DAN: a Segmentation-free Document Attention Network for Handwritten Document Recognition](https://mlsanity.com/paper/2203.12273/) (2022)
- [HTR-VT: Handwritten Text Recognition with Vision Transformer](https://mlsanity.com/paper/2409.08573/) (2024)
- [A Survey of Historical Document Image Datasets](https://mlsanity.com/paper/2203.08504/) (2022)
- [SPAN: a Simple Predict & Align Network for Handwritten Paragraph Recognition](https://mlsanity.com/paper/2102.08742/) (2021)
- [Transformer-Based Approach for Joint Handwriting and Named Entity Recognition in Historical documents](https://mlsanity.com/paper/2112.04189/) (2021)
- [Handwritten Text Recognition: A Survey](https://mlsanity.com/paper/2502.08417/) (2025)
- [Benchmarking Large Language Models for Handwritten Text Recognition](https://mlsanity.com/paper/2503.15195/) (2025)
- [On the Generalization of Handwritten Text Recognition Models](https://mlsanity.com/paper/2411.17332/) (2024)
- [Faster DAN: Multi-target Queries with Document Positional Encoding for End-to-end Handwritten Document Recognition](https://mlsanity.com/paper/2301.10593/) (2023)
- [Key-value information extraction from full handwritten pages](https://mlsanity.com/paper/2304.13530/) (2023)
- [DANIEL: A fast Document Attention Network for Information Extraction and Labelling of handwritten documents](https://mlsanity.com/paper/2407.09103/) (2024)
- [MSdocTr-Lite: A Lite Transformer for Full Page Multi-script Handwriting Recognition](https://mlsanity.com/paper/2303.13931/) (2023)
- [Qalam : A Multimodal LLM for Arabic Optical Character and Handwriting Recognition](https://mlsanity.com/paper/2407.13559/) (2024)
- [Improving Automatic Text Recognition with Language Models in the PyLaia Open-Source Library](https://mlsanity.com/paper/2404.18722/) (2024)
- [A Comprehensive Handwritten Paragraph Text Recognition System: LexiconNet](https://mlsanity.com/paper/2205.11018/) (2022)
- [The Socface Project: Large-Scale Collection, Processing, and Analysis of a Century of French Censuses](https://mlsanity.com/paper/2404.18706/) (2024)
- [End-to-end information extraction in handwritten documents: Understanding Paris marriage records from 1880 to 1940](https://mlsanity.com/paper/2404.19329/) (2024)
- [General Detection-based Text Line Recognition](https://mlsanity.com/paper/2409.17095/) (2024)
- [Phonological Level wav2vec2-based Mispronunciation Detection and Diagnosis Method](https://mlsanity.com/paper/2311.07037/) (2023)
- [Self-Supervised Learning for Text Recognition: A Critical Survey](https://mlsanity.com/paper/2407.19889/) (2024)
- [DocTTT: Test-Time Training for Handwritten Document Recognition Using Meta-Auxiliary Learning](https://mlsanity.com/paper/2501.12898/) (2025)
- [Meta-DAN: towards an efficient prediction strategy for page-level handwritten text recognition](https://mlsanity.com/paper/2504.03349/) (2025)
- [Normalized vs Diplomatic Annotation: A Case Study of Automatic Information Extraction from Handwritten Uruguayan Birth Certificates](https://mlsanity.com/paper/2507.08636/) (2025)
- [StackMix and Blot Augmentations for Handwritten Text Recognition](https://mlsanity.com/paper/2108.11667/) (2021)
- [Revisiting N-Gram Models: Their Impact in Modern Neural Networks for Handwritten Text Recognition](https://mlsanity.com/paper/2404.19317/) (2024)
- [MetaWriter: Personalized Handwritten Text Recognition Using Meta-Learned Prompt Tuning](https://mlsanity.com/paper/2505.20513/) (2025)
- [Handwritten text generation and strikethrough characters augmentation](https://mlsanity.com/paper/2112.07395/) (2021)
- [A Lexicon and Depth-wise Separable Convolution Based Handwritten Text Recognition System](https://mlsanity.com/paper/2207.04651/) (2022)
- [Combining OCR Models for Reading Early Modern Printed Books](https://mlsanity.com/paper/2305.07131/) (2023)

## Abstract (arXiv)

Unconstrained handwritten text recognition remains challenging for computer vision systems. Paragraph text recognition is traditionally achieved by two models: the first one for line segmentation and the second one for text line recognition. We propose a unified end-to-end model using hybrid attention to tackle this task. This model is designed to iteratively process a paragraph image line by line. It can be split into three modules. An encoder generates feature maps from the whole paragraph image. Then, an attention module recurrently generates a vertical weighted mask enabling to focus on the current text line features. This way, it performs a kind of implicit line segmentation. For each text line features, a decoder module recognizes the character sequence associated, leading to the recognition of a whole paragraph. We achieve state-of-the-art character error rate at paragraph level on three popular datasets: 1.91% for RIMES, 4.45% for IAM and 3.59% for READ 2016. Our code and trained model weights are available at https://github.com/FactoDeepLearning/VerticalAttentionOCR.