# TrOCR for Medieval HTR: A Systematic Ablation Study with Cross-Dataset Validation

arXiv: https://arxiv.org/abs/2606.24302 · HTML: https://mlsanity.com/paper/2606.24302/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 29 references that are in the map.
- First posted on arXiv 23 Jun 2026.

## Builds on (most cited first, 10 of 10)

- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Transformer Interpretability Beyond Attention Visualization](https://mlsanity.com/paper/2012.09838/) (2020)
- [LayoutLM: Pre-training of Text and Layout for Document Image Understanding](https://mlsanity.com/paper/1912.13318/) (2019)
- [OCR-free Document Understanding Transformer](https://mlsanity.com/paper/2111.15664/) (2021)
- [TrOCR: Transformer-based Optical Character Recognition with Pre-trained Models](https://mlsanity.com/paper/2109.10282/) (2021)
- [End-to-end Handwritten Paragraph Text Recognition Using a Vertical Attention Network](https://mlsanity.com/paper/2012.03868/) (2020)
- [TRIDIS: A Comprehensive Medieval and Early Modern Corpus for HTR and NER](https://mlsanity.com/paper/2503.22714/) (2025)
- [Optical Music Recognition in Manuscripts from the Ricordi Archive](https://mlsanity.com/paper/2408.10260/) (2024)
- [GatedLexiconNet: A Comprehensive End-to-End Handwritten Paragraph Text Recognition System](https://mlsanity.com/paper/2404.14062/) (2024)
- [Handwritten Text Recognition of Historical Manuscripts Using Transformer-Based Models](https://mlsanity.com/paper/2508.11499/) (2025)

## Built on by (most cited first, 1 of 1)

- [HPMD: A Historical Persian Manuscript Dataset for Word Spotting with Line-Level Annotation](https://mlsanity.com/paper/2609.34490/) (2026)

## Abstract (arXiv)

Fine-tuning transformer-based handwritten text recognition (HTR) models on medieval manuscripts is challenging because these models are pre-trained on modern text and must adapt to a very different visual domain. This paper studies how three controllable fine-tuning choices (contrast normalization, data augmentation, and layer freezing) affect recognition accuracy when adapting TrOCR to small historical datasets. We run controlled experiments on a 13th-century Italian manuscript (I-CT 91 "Cortonese") and replicate the same experimental grid on the public READ-16 benchmark as robustness evidence. On Cortonese, our best configuration achieves 8.03% character error rate (CER). Statistical comparisons across 13 configurations show that freezing up to three encoder layers or six decoder layers does not significantly harm accuracy, while deeper freezing becomes progressively detrimental. Removing contrast normalization (CLAHE) yields 7.84% CER, comparable to a domain-specialized baseline, suggesting strong optimization can reduce reliance on image preprocessing. Cross-dataset validation on READ-16 shows that decoder freezing thresholds transfer more robustly than encoder thresholds, and combined freezing strategies require dataset-specific re-validation. Finally, we use Grad-CAM gradient attributions and decoder cross-attention maps to diagnose error patterns and failure modes revealed by the ablations. Source code is available at https://github.com/LaudareProject/TrOCR-analysis