# Joint Line Segmentation and Transcription for End-to-End Handwritten Paragraph Recognition

arXiv: https://arxiv.org/abs/1604.08352 · HTML: https://mlsanity.com/paper/1604.08352/

## Key facts

- Cited by 46 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Apr 2016.

## Built on by (most cited first, 30 of 46)

- [TrOCR: Transformer-based Optical Character Recognition with Pre-trained Models](https://mlsanity.com/paper/2109.10282/) (2021)
- [End-to-end Document Recognition and Understanding with Dessurt](https://mlsanity.com/paper/2203.16618/) (2022)
- [DAN: a Segmentation-free Document Attention Network for Handwritten Document Recognition](https://mlsanity.com/paper/2203.12273/) (2022)
- [Exploring OCR Capabilities of GPT-4V(ision) : A Quantitative and In-depth Evaluation](https://mlsanity.com/paper/2310.16809/) (2023)
- [Rethinking Text Line Recognition Models](https://mlsanity.com/paper/2104.07787/) (2021)
- [Vectorization and Rasterization: Self-Supervised Learning for Sketch and Handwriting](https://mlsanity.com/paper/2103.13716/) (2021)
- [Full Page Handwriting Recognition via Image to Sequence Extraction](https://mlsanity.com/paper/2103.06450/) (2021)
- [HTR-VT: Handwritten Text Recognition with Vision Transformer](https://mlsanity.com/paper/2409.08573/) (2024)
- [SPAN: a Simple Predict & Align Network for Handwritten Paragraph Recognition](https://mlsanity.com/paper/2102.08742/) (2021)
- [The LAM Dataset: A Novel Benchmark for Line-Level Handwritten Text Recognition](https://mlsanity.com/paper/2208.07682/) (2022)
- [Text-DIAE: A Self-Supervised Degradation Invariant Autoencoders for Text Recognition and Document Enhancement](https://mlsanity.com/paper/2203.04814/) (2022)
- [Boosting Modern and Historical Handwritten Text Recognition with Deformable Convolutions](https://mlsanity.com/paper/2208.08109/) (2022)
- [Seq-UPS: Sequential Uncertainty-aware Pseudo-label Selection for Semi-Supervised Text Recognition](https://mlsanity.com/paper/2209.00641/) (2022)
- [Benchmarking Online Sequence-to-Sequence and Character-based Handwriting Recognition from IMU-Enhanced Pens](https://mlsanity.com/paper/2202.07036/) (2022)
- [How to Choose Pretrained Handwriting Recognition Models for Single Writer Fine-Tuning](https://mlsanity.com/paper/2305.02593/) (2023)
- [Handwritten Text Recognition: A Survey](https://mlsanity.com/paper/2502.08417/) (2025)
- [TextAdaIN: Paying Attention to Shortcut Learning in Text Recognizers](https://mlsanity.com/paper/2105.03906/) (2021)
- [Faster DAN: Multi-target Queries with Document Positional Encoding for End-to-end Handwritten Document Recognition](https://mlsanity.com/paper/2301.10593/) (2023)
- [A Transformer-based Approach for Arabic Offline Handwritten Text Recognition](https://mlsanity.com/paper/2307.15045/) (2023)
- [DANIEL: A fast Document Attention Network for Information Extraction and Labelling of handwritten documents](https://mlsanity.com/paper/2407.09103/) (2024)
- [PageNet: Towards End-to-End Weakly Supervised Page-Level Handwritten Chinese Text Recognition](https://mlsanity.com/paper/2207.14807/) (2022)
- [End-to-End Page-Level Assessment of Handwritten Text Recognition](https://mlsanity.com/paper/2301.05935/) (2023)
- [MSdocTr-Lite: A Lite Transformer for Full Page Multi-script Handwriting Recognition](https://mlsanity.com/paper/2303.13931/) (2023)
- [Auxiliary Cross-Modal Representation Learning with Triplet Loss Functions for Online Handwriting Recognition](https://mlsanity.com/paper/2202.07901/) (2022)
- [Easter2.0: Improving convolutional models for handwritten text recognition](https://mlsanity.com/paper/2205.14879/) (2022)
- [Implicit Feature Alignment: Learn to Convert Text Recognizer to Text Spotter](https://mlsanity.com/paper/2106.05920/) (2021)
- [Lexicon and Attention based Handwritten Text Recognition System](https://mlsanity.com/paper/2209.04817/) (2022)
- [Searching a High-Performance Feature Extractor for Text Recognition Network](https://mlsanity.com/paper/2209.13139/) (2022)
- [Channel-Partitioned Windowed Attention And Frequency Learning for Single Image Super-Resolution](https://mlsanity.com/paper/2407.16232/) (2024)
- [Robust Text Line Detection in Historical Documents: Learning and Evaluation Methods](https://mlsanity.com/paper/2203.12346/) (2022)

## Abstract (arXiv)

Offline handwriting recognition systems require cropped text line images for both training and recognition. On the one hand, the annotation of position and transcript at line level is costly to obtain. On the other hand, automatic line segmentation algorithms are prone to errors, compromising the subsequent recognition. In this paper, we propose a modification of the popular and efficient multi-dimensional long short-term memory recurrent neural networks (MDLSTM-RNNs) to enable end-to-end processing of handwritten paragraphs. More particularly, we replace the collapse layer transforming the two-dimensional representation into a sequence of predictions by a recurrent version which can recognize one line at a time. In the proposed model, a neural network performs a kind of implicit line segmentation by computing attention weights on the image representation. The experiments on paragraphs of Rimes and IAM database yield results that are competitive with those of networks trained at line level, and constitute a significant step towards end-to-end transcription of full documents.