# Towards Robust Handwritten Text Recognition with On-the-fly User Participation

arXiv: https://arxiv.org/abs/2212.08834 · HTML: https://mlsanity.com/paper/2212.08834/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 48 references that are in the map.
- First posted on arXiv 17 Dec 2022.

## Builds on (most cited first, 18 of 18)

- [On The Power of Curriculum Learning in Training Deep Networks](https://mlsanity.com/paper/1904.03626/) (2019)
- [TrOCR: Transformer-based Optical Character Recognition with Pre-trained Models](https://mlsanity.com/paper/2109.10282/) (2021)
- [Dynamic Curriculum Learning for Imbalanced Data Classification](https://mlsanity.com/paper/1901.06783/) (2019)
- [Pay Attention to What You Read: Non-recurrent Handwritten Text-Line Recognition](https://mlsanity.com/paper/2005.13044/) (2020)
- [Evaluating Sequence-to-Sequence Models for Handwritten Text Recognition](https://mlsanity.com/paper/1903.07377/) (2019)
- [Joint Line Segmentation and Transcription for End-to-End Handwritten Paragraph Recognition](https://mlsanity.com/paper/1604.08352/) (2016)
- [Rethinking Text Line Recognition Models](https://mlsanity.com/paper/2104.07787/) (2021)
- [Dropout improves Recurrent Neural Networks for Handwriting Recognition](https://mlsanity.com/paper/1312.4569/) (2013)
- [Accurate, Data-Efficient, Unconstrained Text Recognition with Convolutional Neural Networks](https://mlsanity.com/paper/1812.11894/) (2018)
- [An Efficient End-to-End Neural Model for Handwritten Text Recognition](https://mlsanity.com/paper/1807.07965/) (2018)
- [A Scalable Handwritten Text Recognition System](https://mlsanity.com/paper/1904.09150/) (2019)
- [Recurrence-free unconstrained handwritten text recognition using gated fully convolutional network](https://mlsanity.com/paper/2012.04961/) (2020)
- [Recognition of Handwritten Chinese Text by Segmentation: A Segment-annotation-free Approach](https://mlsanity.com/paper/2207.14801/) (2022)
- [HWNet v2: An Efficient Word Image Representation for Handwritten Documents](https://mlsanity.com/paper/1802.06194/) (2018)
- [Deep Curriculum Learning for PolSAR Image Classification](https://mlsanity.com/paper/2112.13426/) (2021)
- [Fully Convolutional Recurrent Network for Handwritten Chinese Text Recognition](https://mlsanity.com/paper/1604.04953/) (2016)
- [HMM-based Indic Handwritten Word Recognition using Zone Segmentation](https://mlsanity.com/paper/1708.00227/) (2017)
- [Detecting, Tracking and Counting Motorcycle Rider Traffic Violations on Unconstrained Roads](https://mlsanity.com/paper/2204.08364/) (2022)

## Abstract (arXiv)

Long-term OCR services aim to provide high-quality output to their users at competitive costs. It is essential to upgrade the models because of the complex data loaded by the users. The service providers encourage the users who provide data where the OCR model fails by rewarding them based on data complexity, readability, and available budget. Hitherto, the OCR works include preparing the models on standard datasets without considering the end-users. We propose a strategy of consistently upgrading an existing Handwritten Hindi OCR model three times on the dataset of 15 users. We fix the budget of 4 users for each iteration. For the first iteration, the model directly trains on the dataset from the first four users. For the rest iteration, all remaining users write a page each, which service providers later analyze to select the 4 (new) best users based on the quality of predictions on the human-readable words. Selected users write 23 more pages for upgrading the model. We upgrade the model with Curriculum Learning (CL) on the data available in the current iteration and compare the subset from previous iterations. The upgraded model is tested on a held-out set of one page each from all 23 users. We provide insights into our investigations on the effect of CL, user selection, and especially the data from unseen writing styles. Our work can be used for long-term OCR services in crowd-sourcing scenarios for the service providers and end users.