# Practical Fine-Tuning of Autoregressive Models on Limited Handwritten Texts

arXiv: https://arxiv.org/abs/2503.19546 · HTML: https://mlsanity.com/paper/2503.19546/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 34 references that are in the map.
- First posted on arXiv 25 Mar 2025.

## Builds on (most cited first, 14 of 14)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [An End-to-End Trainable Neural Network for Image-based Sequence Recognition and Its Application to Scene Text Recognition](https://mlsanity.com/paper/1507.05717/) (2015)
- [TrOCR: Transformer-based Optical Character Recognition with Pre-trained Models](https://mlsanity.com/paper/2109.10282/) (2021)
- [Pay Attention to What You Read: Non-recurrent Handwritten Text-Line Recognition](https://mlsanity.com/paper/2005.13044/) (2020)
- [Rethinking Text Line Recognition Models](https://mlsanity.com/paper/2104.07787/) (2021)
- [DTrOCR: Decoder-only Transformer for Optical Character Recognition](https://mlsanity.com/paper/2308.15996/) (2023)
- [Boosting offline handwritten text recognition in historical documents with few labeled lines](https://mlsanity.com/paper/2012.02544/) (2020)
- [How to Choose Pretrained Handwriting Recognition Models for Single Writer Fine-Tuning](https://mlsanity.com/paper/2305.02593/) (2023)
- [Mixed Model OCR Training on Historical Latin Script for Out-of-the-Box Recognition and Finetuning](https://mlsanity.com/paper/2106.07881/) (2021)
- [Towards Writing Style Adaptation in Handwriting Recognition](https://mlsanity.com/paper/2302.06318/) (2023)
- [Fine-tuning Is a Surprisingly Effective Domain Adaptation Baseline in Handwriting Recognition](https://mlsanity.com/paper/2302.06308/) (2023)
- [Open Source Handwritten Text Recognition on Medieval Manuscripts using Mixed Models and Document-Specific Finetuning](https://mlsanity.com/paper/2201.07661/) (2022)
- [Mixed Text Recognition with Efficient Parameter Fine-Tuning and Transformer](https://mlsanity.com/paper/2404.12734/) (2024)

## Built on by (most cited first, 1 of 1)

- [Learning to Decipher from Pixels: A Case Study of Copiale](https://mlsanity.com/paper/2604.23683/) (2026)

## Abstract (arXiv)

A common use case for OCR applications involves users uploading documents and progressively correcting automatic recognition to obtain the final transcript. This correction phase presents an opportunity for progressive adaptation of the OCR model, making it crucial to adapt early, while ensuring stability and reliability. We demonstrate that state-of-the-art transformer-based models can effectively support this adaptation, gradually reducing the annotator's workload. Our results show that fine-tuning can reliably start with just 16 lines, yielding a 10% relative improvement in CER, and scale up to 40% with 256 lines. We further investigate the impact of model components, clarifying the roles of the encoder and decoder in the fine-tuning process. To guide adaptation, we propose reliable stopping criteria, considering both direct approaches and global trend analysis. Additionally, we show that OCR models can be leveraged to cut annotation costs by half through confidence-based selection of informative lines, achieving the same performance with fewer annotations.