# How to Choose Pretrained Handwriting Recognition Models for Single Writer Fine-Tuning

arXiv: https://arxiv.org/abs/2305.02593 · HTML: https://mlsanity.com/paper/2305.02593/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 58 references that are in the map.
- First posted on arXiv 4 May 2023.

## Builds on (most cited first, 30 of 30)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Deformable Convolutional Networks](https://mlsanity.com/paper/1703.06211/) (2017)
- [An End-to-End Trainable Neural Network for Image-based Sequence Recognition and Its Application to Scene Text Recognition](https://mlsanity.com/paper/1507.05717/) (2015)
- [TrOCR: Transformer-based Optical Character Recognition with Pre-trained Models](https://mlsanity.com/paper/2109.10282/) (2021)
- [Geometric GAN](https://mlsanity.com/paper/1705.02894/) (2017)
- [Pay Attention to What You Read: Non-recurrent Handwritten Text-Line Recognition](https://mlsanity.com/paper/2005.13044/) (2020)
- [ScrabbleGAN: Semi-Supervised Varying Length Handwritten Text Generation](https://mlsanity.com/paper/2003.10557/) (2020)
- [GANwriting: Content-Conditioned Generation of Styled Handwritten Word Images](https://mlsanity.com/paper/2003.02567/) (2020)
- [OrigamiNet: Weakly-Supervised, Segmentation-Free, One-Step, Full Page Text Recognition by learning to unfold](https://mlsanity.com/paper/2006.07491/) (2020)
- [Joint Line Segmentation and Transcription for End-to-End Handwritten Paragraph Recognition](https://mlsanity.com/paper/1604.08352/) (2016)
- [Scan, Attend and Read: End-to-End Handwritten Paragraph Recognition with MDLSTM Attention](https://mlsanity.com/paper/1604.03286/) (2016)
- [Dropout improves Recurrent Neural Networks for Handwriting Recognition](https://mlsanity.com/paper/1312.4569/) (2013)
- [Adversarial Generation of Handwritten Text Images Conditioned on Sequences](https://mlsanity.com/paper/1903.00277/) (2019)
- [Handwriting Transformers](https://mlsanity.com/paper/2104.03964/) (2021)
- [Text and Style Conditioned GAN for Generation of Offline Handwriting Lines](https://mlsanity.com/paper/2009.00678/) (2020)
- [Content and Style Aware Generation of Text-line Images for Handwriting Recognition](https://mlsanity.com/paper/2204.05539/) (2022)
- [Handwriting Recognition in Low-resource Scripts using Adversarial Learning](https://mlsanity.com/paper/1811.01396/) (2018)
- [MetaHTR: Towards Writer-Adaptive Handwritten Text Recognition](https://mlsanity.com/paper/2104.01876/) (2021)
- [Recurrence-free unconstrained handwritten text recognition using gated fully convolutional network](https://mlsanity.com/paper/2012.04961/) (2020)
- [Boosting offline handwritten text recognition in historical documents with few labeled lines](https://mlsanity.com/paper/2012.02544/) (2020)
- [Rescoring Sequence-to-Sequence Models for Text Line Recognition with CTC-Prefixes](https://mlsanity.com/paper/2110.05909/) (2021)
- [SmartPatch: Improving Handwritten Word Imitation with Patch Discriminators](https://mlsanity.com/paper/2105.10528/) (2021)
- [The LAM Dataset: A Novel Benchmark for Line-Level Handwritten Text Recognition](https://mlsanity.com/paper/2208.07682/) (2022)
- [Boosting Modern and Historical Handwritten Text Recognition with Deformable Convolutions](https://mlsanity.com/paper/2208.08109/) (2022)
- [KuroNet: Pre-Modern Japanese Kuzushiji Character Recognition with Deep Learning](https://mlsanity.com/paper/1910.09433/) (2019)
- [Handwriting Recognition of Historical Documents with few labeled data](https://mlsanity.com/paper/1811.07768/) (2018)
- [Fully Convolutional Networks for Handwriting Recognition](https://mlsanity.com/paper/1907.04888/) (2019)
- [Boosting Handwriting Text Recognition in Small Databases with Transfer Learning](https://mlsanity.com/paper/1804.01527/) (2018)
- [One-shot Compositional Data Generation for Low Resource Handwritten Text Recognition](https://mlsanity.com/paper/2105.05300/) (2021)

## Built on by (most cited first, 13 of 13)

- [HWD: A Novel Evaluation Score for Styled Handwritten Text Generation](https://mlsanity.com/paper/2310.20316/) (2023)
- [Zero-Shot Styled Text Image Generation, but Make It Autoregressive](https://mlsanity.com/paper/2503.17074/) (2025)
- [On the Generalization of Handwritten Text Recognition Models](https://mlsanity.com/paper/2411.17332/) (2024)
- [Binarizing Documents by Leveraging both Space and Frequency](https://mlsanity.com/paper/2404.17243/) (2024)
- [Autoregressive Styled Text Image Generation, but Make it Reliable](https://mlsanity.com/paper/2510.23240/) (2025)
- [Volumetric Fast Fourier Convolution for Detecting Ink on the Carbonized Herculaneum Papyri](https://mlsanity.com/paper/2308.05070/) (2023)
- [Quo Vadis Handwritten Text Generation for Handwritten Text Recognition?](https://mlsanity.com/paper/2508.09936/) (2025)
- [Practical Fine-Tuning of Autoregressive Models on Limited Handwritten Texts](https://mlsanity.com/paper/2503.19546/) (2025)
- [Few-shot Writer Adaptation via Multimodal In-Context Learning](https://mlsanity.com/paper/2603.29450/) (2026)
- [When Low CER is Not Enough: An Analysis of Hallucinations in Vision-Language OCR Systems on Historical Uruguayan Documents](https://mlsanity.com/paper/2607.24077/) (2026)
- [N-gram Injection into Transformers for Dynamic Language Model Adaptation in Handwritten Text Recognition](https://mlsanity.com/paper/2603.03930/) (2026)
- [Zero-Shot Synthetic-to-Real Handwritten Text Recognition via Task Analogies](https://mlsanity.com/paper/2604.09713/) (2026)
- [Measuring Annotation Efficiency for Handwritten Devanagari Recognition: Sample-Complexity Curves for Four Pretraining Regimes](https://mlsanity.com/paper/2609.16859/) (2026)

## Abstract (arXiv)

Recent advancements in Deep Learning-based Handwritten Text Recognition (HTR) have led to models with remarkable performance on both modern and historical manuscripts in large benchmark datasets. Nonetheless, those models struggle to obtain the same performance when applied to manuscripts with peculiar characteristics, such as language, paper support, ink, and author handwriting. This issue is very relevant for valuable but small collections of documents preserved in historical archives, for which obtaining sufficient annotated training data is costly or, in some cases, unfeasible. To overcome this challenge, a possible solution is to pretrain HTR models on large datasets and then fine-tune them on small single-author collections. In this paper, we take into account large, real benchmark datasets and synthetic ones obtained with a styled Handwritten Text Generation model. Through extensive experimental analysis, also considering the amount of fine-tuning lines, we give a quantitative indication of the most relevant characteristics of such data for obtaining an HTR model able to effectively transcribe manuscripts in small collections with as little as five real fine-tuning lines.