# Fine-tuning Is a Surprisingly Effective Domain Adaptation Baseline in Handwriting Recognition

arXiv: https://arxiv.org/abs/2302.06308 · HTML: https://mlsanity.com/paper/2302.06308/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 41 references that are in the map.
- First posted on arXiv 13 Feb 2023.

## Builds on (most cited first, 19 of 19)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [An End-to-End Trainable Neural Network for Image-based Sequence Recognition and Its Application to Scene Text Recognition](https://mlsanity.com/paper/1507.05717/) (2015)
- [TrOCR: Transformer-based Optical Character Recognition with Pre-trained Models](https://mlsanity.com/paper/2109.10282/) (2021)
- [Pay Attention to What You Read: Non-recurrent Handwritten Text-Line Recognition](https://mlsanity.com/paper/2005.13044/) (2020)
- [Evaluating Sequence-to-Sequence Models for Handwritten Text Recognition](https://mlsanity.com/paper/1903.07377/) (2019)
- [Rethinking Text Line Recognition Models](https://mlsanity.com/paper/2104.07787/) (2021)
- [MetaHTR: Towards Writer-Adaptive Handwritten Text Recognition](https://mlsanity.com/paper/2104.01876/) (2021)
- [An Efficient End-to-End Neural Model for Handwritten Text Recognition](https://mlsanity.com/paper/1807.07965/) (2018)
- [Boosting offline handwritten text recognition in historical documents with few labeled lines](https://mlsanity.com/paper/2012.02544/) (2020)
- [Multimodal Semi-Supervised Learning for Text Recognition](https://mlsanity.com/paper/2205.03873/) (2022)
- [Speaker Adaptation for Attention-Based End-to-End Speech Recognition](https://mlsanity.com/paper/1911.03762/) (2019)
- [Adversarial Speaker Adaptation](https://mlsanity.com/paper/1904.12407/) (2019)
- [AT-ST: Self-Training Adaptation Strategy for OCR in Domains with Limited Transcriptions](https://mlsanity.com/paper/2104.13037/) (2021)
- [TS-Net: OCR Trained to Switch Between Text Transcription Styles](https://mlsanity.com/paper/2103.05489/) (2021)
- [Mixed Model OCR Training on Historical Latin Script for Out-of-the-Box Recognition and Finetuning](https://mlsanity.com/paper/2106.07881/) (2021)
- [Self-Training of Handwritten Word Recognition for Synthetic-to-Real Adaptation](https://mlsanity.com/paper/2206.03149/) (2022)
- [Open Source Handwritten Text Recognition on Medieval Manuscripts using Mixed Models and Document-Specific Finetuning](https://mlsanity.com/paper/2201.07661/) (2022)

## Built on by (most cited first, 5 of 5)

- [LoRA Land: 310 Fine-tuned LLMs that Rival GPT-4, A Technical Report](https://mlsanity.com/paper/2405.00732/) (2024)
- [Handwritten Text Recognition: A Survey](https://mlsanity.com/paper/2502.08417/) (2025)
- [Practical Fine-Tuning of Autoregressive Models on Limited Handwritten Texts](https://mlsanity.com/paper/2503.19546/) (2025)
- [Few-shot Writer Adaptation via Multimodal In-Context Learning](https://mlsanity.com/paper/2603.29450/) (2026)
- [OCR-Based Field Extraction for Archaeological Pottery Metadata: The CENTURIA Dataset](https://mlsanity.com/paper/2608.30616/) (2026)

## Abstract (arXiv)

In many machine learning tasks, a large general dataset and a small specialized dataset are available. In such situations, various domain adaptation methods can be used to adapt a general model to the target dataset. We show that in the case of neural networks trained for handwriting recognition using CTC, simple fine-tuning with data augmentation works surprisingly well in such scenarios and that it is resistant to overfitting even for very small target domain datasets. We evaluated the behavior of fine-tuning with respect to augmentation, training data size, and quality of the pre-trained network, both in writer-dependent and writer-independent settings. On a large real-world dataset, fine-tuning on new writers provided an average relative CER improvement of 25 % for 16 text lines and 50 % for 256 text lines.