# Enhancing Handwritten Text Recognition with N-gram sequence decomposition and Multitask Learning

arXiv: https://arxiv.org/abs/2012.14459 · HTML: https://mlsanity.com/paper/2012.14459/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Dec 2020.

## Built on by (most cited first, 11 of 11)

- [Best Practices for a Handwritten Text Recognition System](https://mlsanity.com/paper/2404.11339/) (2024)
- [Classification of Handwritten Names of Cities and Handwritten Text Recognition using Various Deep Learning Models](https://mlsanity.com/paper/2102.04816/) (2021)
- [Improving Automatic Text Recognition with Language Models in the PyLaia Open-Source Library](https://mlsanity.com/paper/2404.18722/) (2024)
- [General Detection-based Text Line Recognition](https://mlsanity.com/paper/2409.17095/) (2024)
- [TrustAI at SemEval-2024 Task 8: A Comprehensive Analysis of Multi-domain Machine Generated Text Detection Techniques](https://mlsanity.com/paper/2403.16592/) (2024)
- [2D Self-Organized ONN Model For Handwritten Text Recognition](https://mlsanity.com/paper/2207.08139/) (2022)
- [Revisiting N-Gram Models: Their Impact in Modern Neural Networks for Handwritten Text Recognition](https://mlsanity.com/paper/2404.19317/) (2024)
- [M-DAIGT: A Shared Task on Multi-Domain Detection of AI-Generated Text](https://mlsanity.com/paper/2511.11340/) (2025)
- [The Challenges of HTR Model Training: Feedback from the Project Donner le gout de l'archive a l'ere numerique](https://mlsanity.com/paper/2212.11146/) (2022)
- [N-gram Injection into Transformers for Dynamic Language Model Adaptation in Handwritten Text Recognition](https://mlsanity.com/paper/2603.03930/) (2026)
- [AEyeDE: An Attention-Based Attribution Framework for AI-Generated Text Detection](https://mlsanity.com/paper/2606.00016/) (2026)

## Abstract (arXiv)

Current state-of-the-art approaches in the field of Handwritten Text Recognition are predominately single task with unigram, character level target units. In our work, we utilize a Multi-task Learning scheme, training the model to perform decompositions of the target sequence with target units of different granularity, from fine to coarse. We consider this method as a way to utilize n-gram information, implicitly, in the training process, while the final recognition is performed using only the unigram output. % in order to highlight the difference of the internal Unigram decoding of such a multi-task approach highlights the capability of the learned internal representations, imposed by the different n-grams at the training step. We select n-grams as our target units and we experiment from unigrams to fourgrams, namely subword level granularities. These multiple decompositions are learned from the network with task-specific CTC losses. Concerning network architectures, we propose two alternatives, namely the Hierarchical and the Block Multi-task. Overall, our proposed model, even though evaluated only on the unigram task, outperforms its counterpart single-task by absolute 2.52% WER and 1.02% CER, in the greedy decoding, without any computational overhead during inference, hinting towards successfully imposing an implicit language model.