# Offline Detection of Misspelled Handwritten Words by Convolving Recognition Model Features with Text Labels

arXiv: https://arxiv.org/abs/2309.10158 · HTML: https://mlsanity.com/paper/2309.10158/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 36 references that are in the map.
- First posted on arXiv 18 Sep 2023.

## Builds on (most cited first, 9 of 9)

- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [Pay Attention to What You Read: Non-recurrent Handwritten Text-Line Recognition](https://mlsanity.com/paper/2005.13044/) (2020)
- [ScrabbleGAN: Semi-Supervised Varying Length Handwritten Text Generation](https://mlsanity.com/paper/2003.10557/) (2020)
- [GANwriting: Content-Conditioned Generation of Styled Handwritten Word Images](https://mlsanity.com/paper/2003.02567/) (2020)
- [Adversarial Generation of Handwritten Text Images Conditioned on Sequences](https://mlsanity.com/paper/1903.00277/) (2019)
- [Text and Style Conditioned GAN for Generation of Offline Handwriting Lines](https://mlsanity.com/paper/2009.00678/) (2020)
- [Content and Style Aware Generation of Text-line Images for Handwriting Recognition](https://mlsanity.com/paper/2204.05539/) (2022)
- [Accurate, Data-Efficient, Unconstrained Text Recognition with Convolutional Neural Networks](https://mlsanity.com/paper/1812.11894/) (2018)
- [Handwriting recognition using Cohort of LSTM and lexicon verification with extremely large lexicon](https://mlsanity.com/paper/1612.07528/) (2016)

## Abstract (arXiv)

Offline handwriting recognition (HWR) has improved significantly with the advent of deep learning architectures in recent years. Nevertheless, it remains a challenging problem and practical applications often rely on post-processing techniques for restricting the predicted words via lexicons or language models. Despite their enhanced performance, such systems are less usable in contexts where out-of-vocabulary words are anticipated, e.g. for detecting misspelled words in school assessments. To that end, we introduce the task of comparing a handwriting image to text. To solve the problem, we propose an unrestricted binary classifier, consisting of a HWR feature extractor and a multimodal classification head which convolves the feature extractor output with the vector representation of the input text. Our model's classification head is trained entirely on synthetic data created using a state-of-the-art generative adversarial network. We demonstrate that, while maintaining high recall, the classifier can be calibrated to achieve an average precision increase of 19.5% compared to addressing the task by directly using state-of-the-art HWR models. Such massive performance gains can lead to significant productivity increases in applications utilizing human-in-the-loop automation.