# Boosting Modern and Historical Handwritten Text Recognition with Deformable Convolutions

arXiv: https://arxiv.org/abs/2208.08109 · HTML: https://mlsanity.com/paper/2208.08109/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 60 references that are in the map.
- First posted on arXiv 17 Aug 2022.

## Builds on (most cited first, 24 of 24)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Deformable Convolutional Networks](https://mlsanity.com/paper/1703.06211/) (2017)
- [Spatial Transformer Networks](https://mlsanity.com/paper/1506.02025/) (2015)
- [An End-to-End Trainable Neural Network for Image-based Sequence Recognition and Its Application to Scene Text Recognition](https://mlsanity.com/paper/1507.05717/) (2015)
- [TrOCR: Transformer-based Optical Character Recognition with Pre-trained Models](https://mlsanity.com/paper/2109.10282/) (2021)
- [Pay Attention to What You Read: Non-recurrent Handwritten Text-Line Recognition](https://mlsanity.com/paper/2005.13044/) (2020)
- [Sequence-to-Sequence Contrastive Learning for Text Recognition](https://mlsanity.com/paper/2012.10873/) (2020)
- [Evaluating Sequence-to-Sequence Models for Handwritten Text Recognition](https://mlsanity.com/paper/1903.07377/) (2019)
- [OrigamiNet: Weakly-Supervised, Segmentation-Free, One-Step, Full Page Text Recognition by learning to unfold](https://mlsanity.com/paper/2006.07491/) (2020)
- [Joint Line Segmentation and Transcription for End-to-End Handwritten Paragraph Recognition](https://mlsanity.com/paper/1604.08352/) (2016)
- [Scan, Attend and Read: End-to-End Handwritten Paragraph Recognition with MDLSTM Attention](https://mlsanity.com/paper/1604.03286/) (2016)
- [Dropout improves Recurrent Neural Networks for Handwriting Recognition](https://mlsanity.com/paper/1312.4569/) (2013)
- [Handwriting Recognition in Low-resource Scripts using Adversarial Learning](https://mlsanity.com/paper/1811.01396/) (2018)
- [Accurate, Data-Efficient, Unconstrained Text Recognition with Convolutional Neural Networks](https://mlsanity.com/paper/1812.11894/) (2018)
- [An Efficient End-to-End Neural Model for Handwritten Text Recognition](https://mlsanity.com/paper/1807.07965/) (2018)
- [Recurrence-free unconstrained handwritten text recognition using gated fully convolutional network](https://mlsanity.com/paper/2012.04961/) (2020)
- [Boosting offline handwritten text recognition in historical documents with few labeled lines](https://mlsanity.com/paper/2012.02544/) (2020)
- [Rescoring Sequence-to-Sequence Models for Text Line Recognition with CTC-Prefixes](https://mlsanity.com/paper/2110.05909/) (2021)
- [KuroNet: Pre-Modern Japanese Kuzushiji Character Recognition with Deep Learning](https://mlsanity.com/paper/1910.09433/) (2019)
- [Fully Convolutional Networks for Handwriting Recognition](https://mlsanity.com/paper/1907.04888/) (2019)
- [TextCaps : Handwritten Character Recognition with Very Small Datasets](https://mlsanity.com/paper/1904.08095/) (2019)
- [Are 2D-LSTM really dead for offline text recognition?](https://mlsanity.com/paper/1811.10899/) (2018)
- [No Padding Please: Efficient Neural Handwriting Recognition](https://mlsanity.com/paper/1902.11208/) (2019)

## Built on by (most cited first, 14 of 14)

- [Handwritten Text Generation from Visual Archetypes](https://mlsanity.com/paper/2303.15269/) (2023)
- [HTR-VT: Handwritten Text Recognition with Vision Transformer](https://mlsanity.com/paper/2409.08573/) (2024)
- [HWD: A Novel Evaluation Score for Styled Handwritten Text Generation](https://mlsanity.com/paper/2310.20316/) (2023)
- [How to Choose Pretrained Handwriting Recognition Models for Single Writer Fine-Tuning](https://mlsanity.com/paper/2305.02593/) (2023)
- [Handwritten Text Recognition: A Survey](https://mlsanity.com/paper/2502.08417/) (2025)
- [DocTTT: Test-Time Training for Handwritten Document Recognition Using Meta-Auxiliary Learning](https://mlsanity.com/paper/2501.12898/) (2025)
- [Autoregressive Styled Text Image Generation, but Make it Reliable](https://mlsanity.com/paper/2510.23240/) (2025)
- [Quo Vadis Handwritten Text Generation for Handwritten Text Recognition?](https://mlsanity.com/paper/2508.09936/) (2025)
- [Dual Orthogonal Guidance for Robust Diffusion-based Handwritten Text Generation](https://mlsanity.com/paper/2508.17017/) (2025)
- [Learning to Align: Addressing Character Frequency Distribution Shifts in Handwritten Text Recognition](https://mlsanity.com/paper/2506.09846/) (2025)
- [DARE: A large-scale handwritten date recognition system](https://mlsanity.com/paper/2210.00503/) (2022)
- [Align, Minimize and Diversify: A Source-Free Unsupervised Domain Adaptation Method for Handwritten Text Recognition](https://mlsanity.com/paper/2404.18260/) (2024)
- [Boosting CNN-based Handwriting Recognition Systems with Learnable Relaxation Labeling](https://mlsanity.com/paper/2409.05699/) (2024)
- [Rethinking Handwritten Character Recognition](https://mlsanity.com/paper/2609.10572/) (2026)

## Abstract (arXiv)

Handwritten Text Recognition (HTR) in free-layout pages is a challenging image understanding task that can provide a relevant boost to the digitization of handwritten documents and reuse of their content. The task becomes even more challenging when dealing with historical documents due to the variability of the writing style and degradation of the page quality. State-of-the-art HTR approaches typically couple recurrent structures for sequence modeling with Convolutional Neural Networks for visual feature extraction. Since convolutional kernels are defined on fixed grids and focus on all input pixels independently while moving over the input image, this strategy disregards the fact that handwritten characters can vary in shape, scale, and orientation even within the same document and that the ink pixels are more relevant than the background ones. To cope with these specific HTR difficulties, we propose to adopt deformable convolutions, which can deform depending on the input at hand and better adapt to the geometric variations of the text. We design two deformable architectures and conduct extensive experiments on both modern and historical datasets. Experimental results confirm the suitability of deformable convolutions for the HTR task.