# An Efficient End-to-End Neural Model for Handwritten Text Recognition

arXiv: https://arxiv.org/abs/1807.07965 · HTML: https://mlsanity.com/paper/1807.07965/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Jul 2018.

## Built on by (most cited first, 25 of 25)

- [TrOCR: Transformer-based Optical Character Recognition with Pre-trained Models](https://mlsanity.com/paper/2109.10282/) (2021)
- [Sheet Music Transformer: End-To-End Optical Music Recognition Beyond Monophonic Transcription](https://mlsanity.com/paper/2402.07596/) (2024)
- [The LAM Dataset: A Novel Benchmark for Line-Level Handwritten Text Recognition](https://mlsanity.com/paper/2208.07682/) (2022)
- [Best Practices for a Handwritten Text Recognition System](https://mlsanity.com/paper/2404.11339/) (2024)
- [Boosting Modern and Historical Handwritten Text Recognition with Deformable Convolutions](https://mlsanity.com/paper/2208.08109/) (2022)
- [Benchmarking Online Sequence-to-Sequence and Character-based Handwriting Recognition from IMU-Enhanced Pens](https://mlsanity.com/paper/2202.07036/) (2022)
- [AttentionHTR: Handwritten Text Recognition Based on Attention Encoder-Decoder Networks](https://mlsanity.com/paper/2201.09390/) (2022)
- [Handwritten Text Recognition: A Survey](https://mlsanity.com/paper/2502.08417/) (2025)
- [A Transformer-based Approach for Arabic Offline Handwritten Text Recognition](https://mlsanity.com/paper/2307.15045/) (2023)
- [TS-Net: OCR Trained to Switch Between Text Transcription Styles](https://mlsanity.com/paper/2103.05489/) (2021)
- [MSdocTr-Lite: A Lite Transformer for Full Page Multi-script Handwriting Recognition](https://mlsanity.com/paper/2303.13931/) (2023)
- [Auxiliary Cross-Modal Representation Learning with Triplet Loss Functions for Online Handwriting Recognition](https://mlsanity.com/paper/2202.07901/) (2022)
- [Easter2.0: Improving convolutional models for handwritten text recognition](https://mlsanity.com/paper/2205.14879/) (2022)
- [Lexicon and Attention based Handwritten Text Recognition System](https://mlsanity.com/paper/2209.04817/) (2022)
- [Searching a High-Performance Feature Extractor for Text Recognition Network](https://mlsanity.com/paper/2209.13139/) (2022)
- [Fine-tuning Is a Surprisingly Effective Domain Adaptation Baseline in Handwriting Recognition](https://mlsanity.com/paper/2302.06308/) (2023)
- [Self-Supervised Learning for Text Recognition: A Critical Survey](https://mlsanity.com/paper/2407.19889/) (2024)
- [2D Self-Organized ONN Model For Handwritten Text Recognition](https://mlsanity.com/paper/2207.08139/) (2022)
- [TransDocAnalyser: A Framework for Offline Semi-structured Handwritten Document Analysis in the Legal Domain](https://mlsanity.com/paper/2306.02142/) (2023)
- [A Lexicon and Depth-wise Separable Convolution Based Handwritten Text Recognition System](https://mlsanity.com/paper/2207.04651/) (2022)
- [A Text Recognition Dataset from Sahidic Coptic Ancient Manuscripts](https://mlsanity.com/paper/2606.15987/) (2026)
- [Towards Robust Handwritten Text Recognition with On-the-fly User Participation](https://mlsanity.com/paper/2212.08834/) (2022)
- [Handwritten Stenography Recognition and the LION Dataset](https://mlsanity.com/paper/2308.07799/) (2023)
- [Improving Handwritten Text Recognition via 3D Attention and Multi-Scale Training](https://mlsanity.com/paper/2410.18374/) (2024)
- [HTR-JAND: Handwritten Text Recognition with Joint Attention Network and Knowledge Distillation](https://mlsanity.com/paper/2412.18524/) (2024)

## Abstract (arXiv)

Offline handwritten text recognition from images is an important problem for enterprises attempting to digitize large volumes of handmarked scanned documents/reports. Deep recurrent models such as Multi-dimensional LSTMs have been shown to yield superior performance over traditional Hidden Markov Model based approaches that suffer from the Markov assumption and therefore lack the representational power of RNNs. In this paper we introduce a novel approach that combines a deep convolutional network with a recurrent Encoder-Decoder network to map an image to a sequence of characters corresponding to the text present in the image. The entire model is trained end-to-end using Focal Loss, an improvement over the standard Cross-Entropy loss that addresses the class imbalance problem, inherent to text recognition. To enhance the decoding capacity of the model, Beam Search algorithm is employed which searches for the best sequence out of a set of hypotheses based on a joint distribution of individual characters. Our model takes as input a downsampled version of the original image thereby making it both computationally and memory efficient. The experimental results were benchmarked against two publicly available datasets, IAM and RIMES. We surpass the state-of-the-art word level accuracy on the evaluation set of both datasets by 3.5% & 1.1%, respectively.