# MetaHTR: Towards Writer-Adaptive Handwritten Text Recognition

arXiv: https://arxiv.org/abs/2104.01876 · HTML: https://mlsanity.com/paper/2104.01876/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 40 of its 52 references that are in the map.
- First posted on arXiv 5 Apr 2021.

## Builds on (most cited first, 30 of 40)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [On First-Order Meta-Learning Algorithms](https://mlsanity.com/paper/1803.02999/) (2018)
- [Meta-Learning in Neural Networks: A Survey](https://mlsanity.com/paper/2004.05439/) (2020)
- [Meta-Learning with Latent Embedding Optimization](https://mlsanity.com/paper/1807.05960/) (2018)
- [TADAM: Task dependent adaptive metric for improved few-shot learning](https://mlsanity.com/paper/1805.10123/) (2018)
- [Meta-SGD: Learning to Learn Quickly for Few-Shot Learning](https://mlsanity.com/paper/1707.09835/) (2017)
- [Rapid Learning or Feature Reuse? Towards Understanding the Effectiveness of MAML](https://mlsanity.com/paper/1909.09157/) (2019)
- [Meta-Dataset: A Dataset of Datasets for Learning to Learn from Few Examples](https://mlsanity.com/paper/1903.03096/) (2019)
- [How to train your MAML](https://mlsanity.com/paper/1810.09502/) (2018)
- [Meta-Transfer Learning for Few-Shot Learning](https://mlsanity.com/paper/1812.02391/) (2018)
- [What Is Wrong With Scene Text Recognition Model Comparisons? Dataset and Model Analysis](https://mlsanity.com/paper/1904.01906/) (2019)
- [Towards Accurate Scene Text Recognition with Semantic Reasoning Networks](https://mlsanity.com/paper/2003.12294/) (2020)
- [Show, Attend and Read: A Simple and Strong Baseline for Irregular Text Recognition](https://mlsanity.com/paper/1811.00751/) (2018)
- [Focusing Attention: Towards Accurate Text Recognition in Natural Images](https://mlsanity.com/paper/1709.02054/) (2017)
- [Decoupled Attention Network for Text Recognition](https://mlsanity.com/paper/1912.10205/) (2019)
- [Recursive Recurrent Nets with Attention Modeling for OCR in the Wild](https://mlsanity.com/paper/1603.03101/) (2016)
- [SCATTER: Selective Context Attentional Scene Text Recognizer](https://mlsanity.com/paper/2003.11288/) (2020)
- [GANwriting: Content-Conditioned Generation of Styled Handwritten Word Images](https://mlsanity.com/paper/2003.02567/) (2020)
- [Learning Meta Face Recognition in Unseen Domains](https://mlsanity.com/paper/2003.07733/) (2020)
- [Tracking by Instance Detection: A Meta-Learning Approach](https://mlsanity.com/paper/2004.00830/) (2020)
- [Meta-Learning via Learned Loss](https://mlsanity.com/paper/1906.05374/) (2019)
- [OrigamiNet: Weakly-Supervised, Segmentation-Free, One-Step, Full Page Text Recognition by learning to unfold](https://mlsanity.com/paper/2006.07491/) (2020)
- [StyleMeUp: Towards Style-Agnostic Sketch-Based Image Retrieval](https://mlsanity.com/paper/2103.15706/) (2021)
- [Symmetry-constrained Rectification Network for Scene Text Recognition](https://mlsanity.com/paper/1908.01957/) (2019)
- [Verisimilar Image Synthesis for Accurate Detection and Recognition of Texts in Scenes](https://mlsanity.com/paper/1807.03021/) (2018)
- [Domain Adaptive Dialog Generation via Meta Learning](https://mlsanity.com/paper/1906.03520/) (2019)
- [DeepWriting: Making Digital Ink Editable via Deep Generative Modeling](https://mlsanity.com/paper/1801.08379/) (2018)

## Built on by (most cited first, 27 of 27)

- [Conditional Text Image Generation with Diffusion Models](https://mlsanity.com/paper/2306.10804/) (2023)
- [Toward Understanding WordArt: Corner-Guided Transformer for Scene Text Recognition](https://mlsanity.com/paper/2208.00438/) (2022)
- [Handwritten Text Generation from Visual Archetypes](https://mlsanity.com/paper/2303.15269/) (2023)
- [Self-supervised Character-to-Character Distillation for Text Recognition](https://mlsanity.com/paper/2211.00288/) (2022)
- [Joint Visual Semantic Reasoning: Multi-Stage Decoder for Text Recognition](https://mlsanity.com/paper/2107.12090/) (2021)
- [HWD: A Novel Evaluation Score for Styled Handwritten Text Generation](https://mlsanity.com/paper/2310.20316/) (2023)
- [VATr++: Choose Your Words Wisely for Handwritten Text Generation](https://mlsanity.com/paper/2402.10798/) (2024)
- [Text is Text, No Matter What: Unifying Text Recognition using Knowledge Distillation](https://mlsanity.com/paper/2107.12087/) (2021)
- [Adaptive Fine-Grained Sketch-Based Image Retrieval](https://mlsanity.com/paper/2207.01723/) (2022)
- [Seq-UPS: Sequential Uncertainty-aware Pseudo-label Selection for Semi-Supervised Text Recognition](https://mlsanity.com/paper/2209.00641/) (2022)
- [How to Choose Pretrained Handwriting Recognition Models for Single Writer Fine-Tuning](https://mlsanity.com/paper/2305.02593/) (2023)
- [Muharaf: Manuscripts of Handwritten Arabic Dataset for Cursive Text Recognition](https://mlsanity.com/paper/2406.09630/) (2024)
- [Handwritten Text Recognition: A Survey](https://mlsanity.com/paper/2502.08417/) (2025)
- [SGBANet: Semantic GAN and Balanced Attention Network for Arbitrarily Oriented Scene Text Recognition](https://mlsanity.com/paper/2207.10256/) (2022)
- [On the Generalization of Handwritten Text Recognition Models](https://mlsanity.com/paper/2411.17332/) (2024)
- [GDB: Gated convolutions-based Document Binarization](https://mlsanity.com/paper/2302.02073/) (2023)
- [Towards Writing Style Adaptation in Handwriting Recognition](https://mlsanity.com/paper/2302.06318/) (2023)
- [Fine-tuning Is a Surprisingly Effective Domain Adaptation Baseline in Handwriting Recognition](https://mlsanity.com/paper/2302.06308/) (2023)
- [APRNet: Attention-based Pixel-wise Rendering Network for Photo-Realistic Text Image Generation](https://mlsanity.com/paper/2203.07705/) (2022)
- [Is it an i or an l: Test-time Adaptation of Text Line Recognition Models](https://mlsanity.com/paper/2308.15037/) (2023)
- [Writer adaptation for offline text recognition: An exploration of neural network-based methods](https://mlsanity.com/paper/2307.15071/) (2023)
- [MetaWriter: Personalized Handwritten Text Recognition Using Meta-Learned Prompt Tuning](https://mlsanity.com/paper/2505.20513/) (2025)
- [Few-shot Writer Adaptation via Multimodal In-Context Learning](https://mlsanity.com/paper/2603.29450/) (2026)
- [Align, Minimize and Diversify: A Source-Free Unsupervised Domain Adaptation Method for Handwritten Text Recognition](https://mlsanity.com/paper/2404.18260/) (2024)
- [DNA: Dual-branch Network with Adaptation for Open-Set Online Handwriting Generation](https://mlsanity.com/paper/2511.22064/) (2025)
- [Tokenization vs. Augmentation: A Systematic Study of Writer Variance in IMU-Based Online Handwriting Recognition](https://mlsanity.com/paper/2603.16883/) (2026)
- [Zero-Shot Synthetic-to-Real Handwritten Text Recognition via Task Analogies](https://mlsanity.com/paper/2604.09713/) (2026)

## Abstract (arXiv)

Handwritten Text Recognition (HTR) remains a challenging problem to date, largely due to the varying writing styles that exist amongst us. Prior works however generally operate with the assumption that there is a limited number of styles, most of which have already been captured by existing datasets. In this paper, we take a completely different perspective -- we work on the assumption that there is always a new style that is drastically different, and that we will only have very limited data during testing to perform adaptation. This results in a commercially viable solution -- the model has the best shot at adaptation being exposed to the new style, and the few samples nature makes it practical to implement. We achieve this via a novel meta-learning framework which exploits additional new-writer data through a support set, and outputs a writer-adapted model via single gradient step update, all during inference. We discover and leverage on the important insight that there exists few key characters per writer that exhibit relatively larger style discrepancies. For that, we additionally propose to meta-learn instance specific weights for a character-wise cross-entropy loss, which is specifically designed to work with the sequential nature of text data. Our writer-adaptive MetaHTR framework can be easily implemented on the top of most state-of-the-art HTR models. Experiments show an average performance gain of 5-7% can be obtained by observing very few new style data. We further demonstrate via a set of ablative studies the advantage of our meta design when compared with alternative adaption mechanisms.