# AT-ST: Self-Training Adaptation Strategy for OCR in Domains with Limited Transcriptions

arXiv: https://arxiv.org/abs/2104.13037 · HTML: https://mlsanity.com/paper/2104.13037/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 23 references that are in the map.
- First posted on arXiv 27 Apr 2021.

## Builds on (most cited first, 6 of 6)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Pointer Sentinel Mixture Models](https://mlsanity.com/paper/1609.07843/) (2016)
- [Improved Regularization of Convolutional Neural Networks with Cutout](https://mlsanity.com/paper/1708.04552/) (2017)
- [Self-training with Noisy Student improves ImageNet classification](https://mlsanity.com/paper/1911.04252/) (2019)
- [An End-to-End Trainable Neural Network for Image-based Sequence Recognition and Its Application to Scene Text Recognition](https://mlsanity.com/paper/1507.05717/) (2015)
- [Page Layout Analysis System for Unconstrained Historic Documents](https://mlsanity.com/paper/2102.11838/) (2021)

## Built on by (most cited first, 10 of 10)

- [Self-Training of Handwritten Word Recognition for Synthetic-to-Real Adaptation](https://mlsanity.com/paper/2206.03149/) (2022)
- [Fine-tuning Is a Surprisingly Effective Domain Adaptation Baseline in Handwriting Recognition](https://mlsanity.com/paper/2302.06308/) (2023)
- [SoftCTC -- Semi-Supervised Learning for Text Recognition using Soft Pseudo-Labels](https://mlsanity.com/paper/2212.02135/) (2022)
- [Self-supervised Pre-training of Text Recognizers](https://mlsanity.com/paper/2405.00420/) (2024)
- [PILOT: A Promptable Interleaved Layout-aware OCR Transformer](https://mlsanity.com/paper/2504.03621/) (2025)
- [Hystoc: Obtaining word confidences for fusion of end-to-end ASR systems](https://mlsanity.com/paper/2305.12579/) (2023)
- [TextBite: A Historical Czech Document Dataset for Logical Page Segmentation](https://mlsanity.com/paper/2503.16664/) (2025)
- [Masked Self-Supervised Pre-Training for Text Recognition Transformers on Large-Scale Datasets](https://mlsanity.com/paper/2503.22513/) (2025)
- [CzechTopic: A Benchmark for Zero-Shot Topic Localization in Historical Czech Documents](https://mlsanity.com/paper/2603.03884/) (2026)
- [Impact of Iterative Fine-Tuning on Transcription Accuracy in Complex Historical Sanskrit Manuscripts](https://mlsanity.com/paper/2608.18696/) (2026)

## Abstract (arXiv)

This paper addresses text recognition for domains with limited manual annotations by a simple self-training strategy. Our approach should reduce human annotation effort when target domain data is plentiful, such as when transcribing a collection of single person's correspondence or a large manuscript. We propose to train a seed system on large scale data from related domains mixed with available annotated data from the target domain. The seed system transcribes the unannotated data from the target domain which is then used to train a better system. We study several confidence measures and eventually decide to use the posterior probability of a transcription for data selection. Additionally, we propose to augment the data using an aggressive masking scheme. By self-training, we achieve up to 55 % reduction in character error rate for handwritten data and up to 38 % on printed data. The masking augmentation itself reduces the error rate by about 10 % and its effect is better pronounced in case of difficult handwritten data.