# Best Practices for a Handwritten Text Recognition System

arXiv: https://arxiv.org/abs/2404.11339 · HTML: https://mlsanity.com/paper/2404.11339/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 33 references that are in the map.
- First posted on arXiv 17 Apr 2024.

## Builds on (most cited first, 13 of 13)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [LSTM: A Search Space Odyssey](https://mlsanity.com/paper/1503.04069/) (2015)
- [Evaluating Sequence-to-Sequence Models for Handwritten Text Recognition](https://mlsanity.com/paper/1903.07377/) (2019)
- [Dropout improves Recurrent Neural Networks for Handwriting Recognition](https://mlsanity.com/paper/1312.4569/) (2013)
- [Learn to Augment: Joint Data Augmentation and Network Optimization for Text Recognition](https://mlsanity.com/paper/2003.06606/) (2020)
- [Accurate, Data-Efficient, Unconstrained Text Recognition with Convolutional Neural Networks](https://mlsanity.com/paper/1812.11894/) (2018)
- [An Efficient End-to-End Neural Model for Handwritten Text Recognition](https://mlsanity.com/paper/1807.07965/) (2018)
- [PHOCNet: A Deep Convolutional Neural Network for Word Spotting in Handwritten Documents](https://mlsanity.com/paper/1604.00187/) (2016)
- [A Fully Differentiable Beam Search Decoder](https://mlsanity.com/paper/1902.06022/) (2019)
- [Enhancing Handwritten Text Recognition with N-gram sequence decomposition and Multitask Learning](https://mlsanity.com/paper/2012.14459/) (2020)
- [Cells in Multidimensional Recurrent Neural Networks](https://mlsanity.com/paper/1412.2620/) (2014)

## Built on by (most cited first, 17 of 17)

- [One-Shot Diffusion Mimicker for Handwritten Text Generation](https://mlsanity.com/paper/2409.04004/) (2024)
- [DiffusionPen: Towards Controlling the Style of Handwritten Text Generation](https://mlsanity.com/paper/2409.06065/) (2024)
- [On the Generalization of Handwritten Text Recognition Models](https://mlsanity.com/paper/2411.17332/) (2024)
- [Beyond Isolated Words: Diffusion Brush for Handwritten Text-Line Generation](https://mlsanity.com/paper/2508.03256/) (2025)
- [Rethinking HTG Evaluation: Bridging Generation and Recognition](https://mlsanity.com/paper/2409.02683/) (2024)
- [A Human-in-the-Loop Label Error Detection Framework Applied to Arabic-Script HTR Datasets](https://mlsanity.com/paper/2601.16713/) (2026)
- [Quo Vadis Handwritten Text Generation for Handwritten Text Recognition?](https://mlsanity.com/paper/2508.09936/) (2025)
- [Dual Orthogonal Guidance for Robust Diffusion-based Handwritten Text Generation](https://mlsanity.com/paper/2508.17017/) (2025)
- [Cross-Lingual Learning within Arabic Script for Low-Resource HTR](https://mlsanity.com/paper/2605.02089/) (2026)
- [CTC Transcription Alignment of the Bullinger Letters: Automatic Improvement of Annotation Quality](https://mlsanity.com/paper/2508.07904/) (2025)
- [Handwritten Text Recognition of Historical Manuscripts Using Transformer-Based Models](https://mlsanity.com/paper/2508.11499/) (2025)
- [Understanding Cross-Language Transfer Improvements in Low-Resource HTR: The Role of Sequence Modeling](https://mlsanity.com/paper/2605.05900/) (2026)
- [Boosting CNN-based Handwriting Recognition Systems with Learnable Relaxation Labeling](https://mlsanity.com/paper/2409.05699/) (2024)
- [HTR-JAND: Handwritten Text Recognition with Joint Attention Network and Knowledge Distillation](https://mlsanity.com/paper/2412.18524/) (2024)
- [DNA: Dual-branch Network with Adaptation for Open-Set Online Handwriting Generation](https://mlsanity.com/paper/2511.22064/) (2025)
- [BullingerDB: A Dataset for Handwritten Text Recognition and Writer Retrieval](https://mlsanity.com/paper/2605.30235/) (2026)
- [Performance Gap Analysis between Latin and Arabic Scripts HTR](https://mlsanity.com/paper/2606.18884/) (2026)

## Abstract (arXiv)

Handwritten text recognition has been developed rapidly in the recent years, following the rise of deep learning and its applications. Though deep learning methods provide notable boost in performance concerning text recognition, non-trivial deviation in performance can be detected even when small pre-processing or architectural/optimization elements are changed. This work follows a ``best practice'' rationale; highlight simple yet effective empirical practices that can further help training and provide well-performing handwritten text recognition systems. Specifically, we considered three basic aspects of a deep HTR system and we proposed simple yet effective solutions: 1) retain the aspect ratio of the images in the preprocessing step, 2) use max-pooling for converting the 3D feature map of CNN output into a sequence of features and 3) assist the training procedure via an additional CTC loss which acts as a shortcut on the max-pooled sequential features. Using these proposed simple modifications, one can attain close to state-of-the-art results, while considering a basic convolutional-recurrent (CNN+LSTM) architecture, for both IAM and RIMES datasets. Code is available at https://github.com/georgeretsi/HTR-best-practices/.